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ABSTRACT 
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of the 1993-94 school year, the Instructional Product Evaluation 
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evaluation of "Konichi Wa," The purpose of this study was to provide 
formative evaluation findings to bx Incorporated into producing the 
next language videodisc package, the Chinese-Korean videodisc. The 
evaluation procedure was divided into two approaches: to determine if 
the materials functioned ps intended and to determine how effective 
the materials are when used in a classroom. The results of the 
evaluation are presented in the following categories: teaching 
materials; videodisc; cultural materials; and on-site visit. Findings 
showed that this approach to teaching foreign languages has received 
enthusiastic implementation and the videodisc works as intended. 
Included in the appendixes are the full evaluation reports from each 
component and the evaluation instruments, (JLB) 
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ABSTRACT 

Konichi-Wa is designed to teach Japanese language and culture via a multime- 
dia videodisc, a teacher's guide, and supplemental cultural materials. The producers 
objectives are to provide a learning experience for children grades K-6 with instruction 
coming from aregular classroom teacher who has had little or no formal language 
instruction training in foreign language training. 

During the winter quarter of the 1993-1994 school year, the Instructional Product 
Evaluation class (Inst T 679) taught by Dr. Nick Eastmond was asked to participate in 
the evaluation of the Japanese Videodisc, Konichi Win. The evaluation was conducted 
as a joint venture under the auspices of ths Worldwide Education and Research 
Institute of Salt Lake City, Utah. The evaluation course is part of the curriculum in the 
Instructional Technology Department at Utah State University. The videodisc and its 
accompanying supplementary material were produced by the Utah Videodisc 
Consortium ( Art Burnah, Larry Gale, et al.). 

Our task as evaluators was to provide formative evaluation findings that can bo 
incorporated into producing the next language videodisc package, the Chinese-Korean 
videodisc. The evaluation was designed to answer three questions: 1 ) Does the package 
function properly, 2) Can it teach students, and 3) Does the program have interest and 
appeal for students and instructors? The emphasis of the evaluation will be to provide 
suggestions for modification of the present implementation and lessons for use in the 
future videodisc products. 
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The findings to date are impressive. This approach to foreign language teaching, 
a radical departure from traditional language instruction, has received enthusiastic 
implementation in Clark County, Nevada and has been well received for the first phase 
of implementation among elementary teachers in Utah. While it is premature to obtain 
data regarding implementation at the elementary level has not been generated 
(implemetation for Utah schools begins in September, 1994), results to date with teach- 
ers and students in Clark County and with teachers in Utah have been most promising. 

INTRODUCTION 

As background information, it is important to note that the present Japanese 
videodisc was the successor to previous videodisc development done by the Utah 
Videodisc Consortium, with team members funded by the Provo and Alpine School 
Districts , the Utah State Office of Education and federal grants. 

A previous product completed by the Utah Videodisc Consortium, Hablar et 
Parler, designed — as the name implies — to teach both Spanish and French, used a sim- 
ilar approach for use at the elementary school level. The essence of this approach is to 
use authentic speech filmed in- country on the videodisc to allow elementary teachers 
with no previous language teaching experience to teach and learn along with their stu- 
dents. Previous evaluation work (Eastmond, Samhouri, and Durrant, 1993) attested to 
the effectiveness of that approach. 

To accomplish this task, evaluation teams chose to use a multiple methods 
approach to evaluate the effectiveness of the product (Worthen and Sanders, 1987). 
Two broad areas were defined for this evaluation. The first involved the entire class of 
26 students, divided into three groups with the tasks of evaluating the different por- 
tions of the actual product The groups looked at the videodisc itself, the accompanying 
teaching materials, and lastly, the cultural materials. Each group was then subdivided 
into two teams that were to formulate their own perspectives at looking at the assigned 
materials. When the investigations were complete the findings of the two teams were 
synthesized into a formal report incorporating both teams findings. 

The teams were assigned to examine the materials to determine if they func- 
tioned as intended. Questions such as the following were asked nbout the videodisc 
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material: Were the elements of the package functioning correctly? Could the audio and 
video segments be understood? Did the bar codes in the teaching materials work and 
did they cue the videodisc to the correct segment? 



The second area of this evaluation involved the usag e of the material. A sub- 
group of the class was selected to look at the implementation of the material. The Utah 
State Board of Education implemented an introductory test of Konichi Wa through 
inservice orientations for perspective teachers of the material and then moving to a trial 
period of instruction. This trial involved a volunteer sample of schools within Utah 
that would include as many different demographic areas as possible. A report of the 
Japanese video disc inservice (conducted over the state's microwave transmission sys- 
tem, ED-NET) is included in Appendix F. To evaluate the inservice program, three 
means of data collection were adopted: The first approach was participant observation, 
the second approach a questionnaire survey at the completion of training, and the third 
approach was telephone survey conducted 2 months later. 

During 1993-94 only Clark County, Nevada was using Konichi Wa. The state of 
Nevada has been divided into 17 educational districts that coincide with the 17 coun- 
tries in Nevada. Clark County School District is the largest in the state and contains 
Nevada's largest city, Las Vegas, and represents an ethnically and economically diverse 
population. 

Four people making the site visits to Las Ve^as divided into two teams to gather 
data on the usage of Konichi Wa . The teams took video recordings of classroom inter- 
actions with the videodisc. A video that details the usage of the videodisc format is 
now being prepared. The video will use footage from the Clark County interviews and 
also from the teacher inservice in Utah. 

Methods 

The evaluation was carried out by dividing the evaluation into two major areas 
of effort. After interviewing Joan Patterson, the Utah State Board of Education foreign 
language specialist, and Art Burnah, a French teacher at Provo High School and lead 
developer of the videodisc a set of evaluation criteria was established that would to 
focus evaluation efforts. The questions to be investigated fall into three broad cate- 
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gories: 

1 Mechanics: Does the package work? 

2 Teaching: Can Konichi Wa teach students? 

3 Does the program have interest and appeal? 



Each group within the major areas used a different approach to their phase of the eval- 
uation. The methods section will be divided into subsections that detail each facet's 
evaluation techniques. 
Teaching Materials 

The two teams involved in this phase of the evaluation used the objectives-ori- 
ented approach and to some extent the connoisseur-based approach to the teaching 
materials that were included with the videodisc (Flagg, 1990). To perform the objec- 
tives-oriented approach the teams checked the printed materials' stated objectives and 
its correspondence to actual course content as further outlined in the teaching materi- 
als. The groups tried to determines if the teaching materials further clarified the goals 
of the course. 

All of the evaluators are graduate students in Instructional Technology. 
Although limited in practical experience in instructional media and design, they come 
h om a diverse background and have received the current training needed for a con- 
noisseur-based evaluation. For this portion of the evaluation the teams looked at the 
teaching materials setup. Specifically, they examined areas of illustration, presentation 
and referencing of important points, and at the teachers explanations of the lesson's 
content Both teams also examined the bar code setup for ease of use as well as ease in 
finding a given segment of the videodisc. 



The Videodisc 



For the evaluation of the videodisc the two groups took slightly different 
approaches. Here the teams were concerned not only with the material presented in the 
lessons but also in the actual use of the videodisc and the videodisc player. For this por- 
tion of the evaluation the teams adopted a decision-oriented approach again, but aug- 
mented this approach with a participant-oriented approach. 
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The teams wanted to find out if the material contained on the videodisc could be 
understood and used for effective instruction. At the same time they wanted to know if 
the medium used to convey the instruction was easy to use and produced a consistent 
outcome when queried. 

Both teams viewed the videodisc in the role of learners and/or first time instruc- 
tors. They attempted to determine if the audio and video were intelligible and if materi- 
al was presented at a speed that could be understood by new learners. The groups then 
gave a concerted effort to determining whether the bar codes worked correctly. To 
accomplish this end, the groups devised a number of repetitive experiments to see how 
well the bar code reader was working and how accurately it queried the videodisc. 

The videodisc groups next turned to the question of whether the material could 
teach Japanese language. The groups participated in viewing all the lessons. Each les- 
son was evaluated on whether the bar codes accurately portrayed the lessons objec- 
tives. They also cross-referenced the printed bar codes to the strategy sections of the 
teaching materials. The cross-referencing was used to determine if the lessons' bar code 
segments were augmented by the teaching material or had any referencing at all. 

Both groups, then, concentrated on the lessons content. Here they tried to 
answer questions on how well the videodisc can teach the Japanese language and cul- 
ture. They specifically wanted to know if using the bar codes made the course more 
flexible to the teacher and were the lessons appropriate for 45-50 minute sessions. 

Supplemental Cultural Materials 

The supplemental cultural materials were produced by the Clark County School 
District, Nevada. Although they appear rot to be copyrighted at this time, there has 
been no formal indication that this material will be supplied along with the videodisc to 
future purchasers. Both groups evaluating this material state that it would be most ben- 
eficial to the objectives of Kotiichi Wa to have them included. 
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The two evaluation groups chose to use the objectives-oriented and the decision- 
oriented approaches to the cultural material. To accomplish their task they broke their 
evaluation into six sections: 

1 Global analysis 

2 Content 

3 Instructional Design 

4 Record Keeping 

5 Ease of Use 

6 Conclusions. 

Each section was then represented using a two-dimensional matrix. A value judgement 
scale is portrayed horizontally and the subcategories of the evaluation in the vertical 
direction. Further, specific commentary was provided to substantiate their rating of the 
different subcategories. 

The teams then looked at each cultural lesson individually and built a two- 
dimensional matrix that evaluated eich lesson on four aspects of the lesson: 

1 Objectives 

2 Materials 

3 Background Information 

4 Activities. 

Each lesson was viewed and the overall comments of the evaluators were placed in the 
appropriate box of the matrix. 

Clark County Observations 

This is the first group that has been able to complete an evaluation of an actual 
implementation of the Japanese videodisc in the classroom. For this phase of the evalu- 
ation two groups with two members each were sent to Clark County, Nevada. Each 
group was assigned to conduct an evaluation at a separate middle school. At the time 
of the evaluation nine schools in Clark County had adopted Konichi Wa as their 
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Japanese language courseware. Only two of the schools taught more than one class 
using the material. Elena Steel, the district foreign language curriculum specialist in 
Clark County, felt that the two multiple class schools would best suit the purpose of the 
evaluation. She made arrangements for our two groups to conduct their evaluations at 
these schools. 

Both schools chosen, On Middle School and Gibson Middle School, teach grades 
6 through 9. Most children attending these schools are in the 1 1 to 15 age groups. The 
school's students come from diverse multi-cultural backgrounds. In terms of socioeco- 
nomic status (SES), these students come from predominantly middle to lower-middle 
class families. It should be noted that two features of the Clark County implementation 
are different than originally proposed: 

1 The materials are being implemented at the middle school rather than at the 
elementary school level. 

2 The teachers are experienced foreign language teachers whose prior experi- 
ence has been in a language other than Japanese (e.g. Spanish, French or 
German). Thus, the interpretation of the following findings must be tem- 
pered when considering the materials use in the elementary school classroom 
by regular classroom teachers with no foreign language teaching experience. 

Our on-site evaluators took an objectives-oriented evaluation approach that used 
vario is methods to determine the effectiveness of the Japanese videodisc: 

1 Classroom observation 

2 Student focus groups 

3 Instructor interviews 

4 Instructor written surveys 

5 Interviews with principals and the district specialist 

At the Orr Middle School, the classes that were evaluated were sixth grade. At 
Gibson the classes were seventh grade. During the observation periods the students 
were evaluated on their attitudes towards the material, their accomplishments and lan- 
guage skills, and whether the learning strategies of the video disk were effective. 
At the same time the two groups videotaped the students both in class and in their 
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focus groups. This footage wiil be edited and used in the accompanying video that will 
be delivered with the final evaluation report. 

RESULTS 

This section will be subdivided into the two major areas of the evaluation plan: 

1 Do the provided materials perform their intended 
function? 

2 How effective is the Japanese videodisc when used in a 
classroom situation? 

This method of reporting the results has been chosen because without the completion 
of Utah inservice word and implementation at the elementary school level, the class- 
room effectiveness of the Konichi Wa program cannot fully be examined. 

The Materials Intended Function 

Teaching Materials 

The group involved with the teaching materials found that the objectives were 
well defined and the materials led the instructor to a means of accomplishing the objec- 
tives. They frvnd that icons designed to emphasize certain points of the lessons ( !, FYI, 
and *) weic weu ^d, allowing instructors to get to the specific information that they 
might need in an expedient manner. They further found that the need for language 
reinforcement was included. The teaching materials w<ire judged as doing a good job of 
identifying the Japanese phonetics that are not in common with English phonetics. 

The teaching materials group suggests the following modifications could be 
made to augment and improve the teaching materials: 

1 An indication of the amount of lesson preparation time would be helpful for a 
new instructor. 

2 The various learning themes could be tabbed and indexed for easier access to 
the material. This should include an index indicating where to find a particu- 
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lar passage's barcode. 

3 A short troubleshooting section would be handy to help alleviate some of the 
anxiety of using this type of instructional media. (Presumably this material 
would also be covered in the inservice explanations.) 

4 At present the video does not do a consistent job of presenting the Japanese 
characters that represent numbers, days of the week, months of the year, and 
the colors. Improving the consistency of these characters use would help. 

5 The group feels that degree of language formality may be atypical. They point 
out that the language is quite fornv..i, presented as one-way conversation (e.g. 
no conversational interaction).Th<?y further state that few "action" verbs are 
taught and the dialog may initially be spoken too quickly. 

Please refer to Appendix A for further clarification of the findings of the Teaching 
Materials evaluation group. 

Videodisc 

Upon reviewing the actual videodisc's contents and method of access, this 
group's evaluation states: 

1 The choice of background and settings aid in interpreting the speech of the 
Japanese children and reinforce the content of the lessons. 

2 The bar codes and the bar code readers work very well, and the levei of 
"successful reads" is adequate for classroom instruction. 

3 By running through a number of the instruction summaries, it was determined 
the lesson would be effective and accomplishable using the 45-50 minute 
instruction period. The subdividing of the lessons into sections further aids in 
maintaining appropriate sized lessons. 

4 The group feels that Japanese life-style is covered well but this aspect of the 
instruction might be incorporated earlier in the lesson sequence. 

The videodisc group also has a number of recommendations: 

1 They feel that at times there is too much background noise which interferes 
with the linguistic instruction. 
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2 The camera work, although quite professional, is somewhat static in use of 
composition and camera angles.They also point out that there are a number of 
sections where the video "flickers" during motion. 

3 Japanese children tended to speak too fast for non Japanese speaking students 
to understand, especially at first. The group points out that this condition 
tends to give the appearance of garbled syllables and frequently garbled words. 

4 The videodisc evaluators also noted a number of inconsistencies between the 
strategy section of the teaching materials and the bar code section. Frequently 
a section on lesson strategy details a number of bar code segments where there 
may be a different number (more or less) of bar codes accessible from that 
page. 

5 A number of lessons that build upon previous lessons are inconsistent in 
usage cf the videodisc. Lessons 3 and 4 on numbers are a prime example. 

6 The . is a need for more detail as to what the instructor will receive 

with each bar code usage (e.g. there is no notification when video or audio are 
not present for a particular segment) and there are no captions at all for some 
segments. 

7 Lesson 24 is very confusing. The colors shown on the screen are not the colors 
specified in the guidebook. 

8 The use of animation in lessons 48, 49, and 50 is disappointing. The prior 
lessons are very impressive, but this group of lessons uses animation that is 
not well conceptualized and wa* 'udged less than adequate on execution. 

9 Lastly, the lessons are formatted on the laserdisc in an order that is sometimes 
inconvenient for the teacher. Currently the instructor must pay close attention 
to which side of the videodisc (side 1 or 2) the lesson is on and may have to 
continually cue different sides of the disc to maintain the ordered sequence. 

Please refer to Appendix B for further clarification of the videodisc evaluation group's 
findings. 

Cultural Materials 

The supplemental Cultural Materials were developed by the Clark County 
School District for incorporation into the curriculum of the course. The unanimous 
feeling is that this cultural material is important to include. The evaluators found that 
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the content and the instructional design were quite thorough and would be a tremen- 
dous aid in developing awareness and teaching respect for the Japanese culture. 
Unfortunately, due to non-concurrent development, the cultural materials are not 
linked closely to the videodisc lessons. The material is good enough, however, that it 
could be used as a stand-alone teaching package. The units offer a variety of activities 
that can potentially keep the interest level of the students high. 

In critiquing the cultural materials, the evaluation group expressed the following 
concerns: 

1 There is no introduction nor set of instructions for usage connecting the cul- 
tural materials and the videodisc lessons. This omission is probably due to the 
nature of the development of this material independently from the videodisc 
material. 

2 The materials touch on different areas of Japanese culture but there is no 
attempt to develop the relationships that exist between the different areas. 

3 There is no integrated assessment system for the cultural material and no indi 
cators as to how the authors feel student achievement in this could be 
assessed. 

4 The evaluators feel that many parts of the Japanese culture affect language 
and non-verbal communications. For example, the early explanation of cus- 
toms of bowing is most important. There is not enough development of this 
theme and in many cases it does not tie in to the videodisc material For 
example, seeing the various forms of bowing as done on the videodisc would 
be an important link. 

Please refer to Appendix C for further clarification of the Supplemental Cultural 
Materials evaluation* 

On-Site Visit to Clark County 

The Clark County School District 's on-site evaluation tends to reinforce the 
opinions expressed by the previous evaluation groups. Both the instructors and the stu- 
dents indicated that the videodisc format is instructive and entertaining at the same 
time. The students appear to pay close attention to the video and are able to relate well 
to many of the characters. Both instructors indicate in their interviews that the video 
language instruction is well received by students, but that they must limit its usage to 
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15 - 20 minutes per day or risk losing the students' attention. 

Both the instructors and the students feel that the ability for immediate replay of 
a segment that they do not understand is one of the greatest benefits of the videodisc 
format The ability to actually hear the words spoken by a native Japanese speaker is 
also seen as an aid to recognition. The students feel that they have a model that they 
can imitate. 

The instructors both indicated having enjoyed the experience of learning the lan- 
guage along with the students. For them the videodisc has surpassed all expectations 
as a medium for teaching a new foreign language. Incidentally, both instructors indi- 
cated that they want to teach Japanese for a longer period than nine weeks and wish to 
become accredited in Japanese to do so. 

Our in-class evaluators indicated that in all four classes that they observed, all 
the students appeared to understand the lessons and that a very consistent outcome 
was being elicited from the entire class. This is very encouraging since Mr. West, the 
Japanese teacher at Gibson Middle School, indicated that he not only has highly moti- 
vated students in his classes but also a number of special education students are in his 
morning class. 

The on-site evaluations do have a few recommendations "from the field": 

1 Everyone involved believes that drawing upon the cultural material is a neces- 
sity ir, teaching languages. Especially this linkage is important for Japanese, 
since its language and culture are so deeply rooted in Japanese traditions. 

2 The students indicated that the language was hard at first to understand, but 
as they got used to it, the speed was not such a problem. Interestingly, the stu- 
dents indicated the characters that they disliked were the ones that either 
talked too fast or too quietly and could not be understood well. 

3 A major concern with both instructors is the inconsistency of the volume level. 
This is mentioned most as the greatest technical difficulty with the videodisc. 

4 Another technical difficulty that was expressed was the inability to scan the 
bar codes properly when the batteries in the scanner became low. Ms. Arroyo 
indicates that she must purchase her own batteries when they are needed. 
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5 Both teachers indicate their opinion that the course needs to be taught by a 
language specialist but not necessarily a Japanese language specialist. (As 
noted above, this use of the videodisc by foreign language teachers is different 
than that originally proposed) 

6 The students in two focus groups indicated that they would like to see the 
written materials for students (e.g. worksheets and project notes) bound 
together so they have less chance of losing them. That form of packaging 
would help be better organized. 

7 Patricia Arroyo, Orr Middle School, states that "the activities need to be on the 
same page as the bar code so I do not have to flip back and forth." Presumably 
she she is referring to the cultural materials taught in conjunction with the 
teaching materials. 

The inservice session for elementary level teachers in Utah was conducted over ED- 
NET telecommunication system (microwave transmission) on three Friday afternoon 
sessions in April and May, with classroom implementation scheduled for fall 1994 in 
Utah. As a result of questionnaires from participants (25), and follow-up telephone 
interviews with 10 persons one month after training, it is clear that the orientation was 
well received and viewed as professionally done. The strategy of using a technical rep- 
resentative at the various receiving sites to supervise "hands on" practice was seen as 
being partially effective. The participant 7 s only concern, and thus concern was 
expressed by fewer than half of respondents, was the use and implementation of equip- 
ment in the classroom. Please refer to Appendix D for further information on the 
Gibson Middit. School evaluation and to Appendix E for the Orr Middle School evalua- 
tion. 



CONCLUSIONS 

The major finding — after many hours of interviewing, observing, analyzing and 
reflecting upon the results — the Konichi-Wa videodisc works as intended. Given the 
right equipment and training, teachers can use it in classrooms and students both enjoy 
the program and learn from it. 
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Konichi-Wa represents a radical departure from traditional language learning in 
the United States in being technology-based, starting in the elementary school, and pre- 
senting language and culture far removed from the majority of its intended audience. 
As evidenced by the enthusiastic reception of teachers in Utah and Nevada, the entire 
teaching/ learning package comes across as polished, professional, and an effective way 
to learn the rudiments of the Japanese language. 

The evaluators feel a certain amount of trepidation with the on-site evaluations 
that have been conducted so far. The Konichi Wa videodisc's objectives clearly state 
that this package is intended for grades K-6 using a non-language specialist as the 
instructor. While it is true that Ms. Arroyo's classes were in the sixth grade, this is in a 
middle-school atmosphere where each teacher only handles a particular class for one 
hour a day. Mr. West's class was made up of seventh graders. Both instructors are for- 
eign language specialists. The evaluators feel that the on-site evaluations so far, how- 
ever, have been impressive and suggest that when the in-service training in Utah is 
complete, and when the materials have been put to use in elementary classrooms we 
can anticipate having additional information to evaluate on the basi* of the videodisc's 
stated objectives. But the results thus far are most encouraging. 

The evaluators feel that the Japanese Videodisc Konichi Wa provides a valuable 
resource for foreign language education. We found it effective as a teaching package 
maintaining an interest level with the students that is higher than most packages of its 
kind. Although at this point our measures of the levels of competency and retention of 
material are not quantitative, every indication points to this format as being highly 
effective. Students with no previous experience with foreign language were observed 
enthusiastically responding in Japanese. The instructors indicate that this mode of 
instruction gives a greater level of consistency among students than some traditional 
methods. Mr. West's comment that the motivated students will take better notes while 
using the videodisc and the remedial students' attention is kept focused better by the 
videodisc indicates effectiveness of this package. 

The evaluators feel that the findings of the materials evaluators' resulting recom- 
mendations are born out rather well from the on-site observations. Indeed, many of the 
evaluators recommendations and observations are echoed both by the Clark County 
instructors and by their students in the focus group discussions. 
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Introduction 



In summary, we feel the content is well thought out and for the most part well 
organized. Viewed from the perspective of the total teaching package, our suggestions 
for improvement are minor ones. We found slight problems with audio volume and 
enunciation for some speakers. Also some additional organization of the teaching 
material (e.g. an index, a bar code, sequential access to the lessons - without having to 
turn the videodisc over, and better linkages between cultural and videodisc segments) 
would improve the product. The instructional approach taken in Konichi Wa has the 
potential to become the exemplary material that other interactive teaching material 
could t>; modeled after. It has the potential of making a major impact on language 
learning at the elementary level, with spin-offs going well beyond, into the middle 
school and high school, opening our schools to more widespread influence of far 
Eastern languages and cultures. 
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Teaching Materials 



A Word of Explanation 

The process of combining the observations and experiences each of the team 
members had been demanding. Each member has had different experiences in learning 
languages, in teaching children about languages, and each has different concepts about 
how children learn languages as well as how they should be taught This reporting 
combines these experiences into several major points, both good and bad, that came up 
repeatedly in the observations. Each student's ideas have been included in examining 
these points. 

General Comments 

The observation was made that initially the interest in the videodisc would be 
good and keep the students attention, but that as time passed maintaining that interest 
could be a problem. It was also observed that initially, the time required to teach using 
the \ ideo disc would be very high, but that over time the preparation time would 
decrease. 

Favorable Impressions 

Introductory Information 

The information available is adequate for most lessons. 
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A. The information given is easy to use. 

B. The icons were good, and give the text a friendly look. 

1. There were some concerns after using the program, 

that the icons denoting "attention", etc. were not as effective as 
first thought. 

C. The objectives were well defined. 

Well illustrated 

A. The disc and the teaching manual were well done. 

B. There could have been more graphics in the manual. 

1) The following lessons had no graphics and could have 
used them to teach more effectively and keep student inter 
est better: 9, 10, 23, 25, 29, 35, 39, 40, 41, 47, 53, 57. 

2) Fewer than half of the lessons had graphics that the team 
considered ample and appropriate. Particularly the review 
lessons may have been inadequate. 

Important areas were flagged 



A. The !, FY1, and *, symbols were used well 

B. It was suggested that the material be tabbed with the various learning 
themes, for more rapid access. 



Teacher explanations are easy to use 

A. The need to review and re-review was anticipated and written in. 

B. The program was responsive to the making of the sounds used in 
Japanese that are not used in English. 

C. The structure is open with good use of 'white space 1 . 

D. The teacher may need better phonetic illustrations to understand what is 
^oing on in the lesson. 



Functional Baals 
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A. The lessons were flexible while still allowing the teacher to accomplish 
the goals set forth in the material. 

Areas For Possible Improvement 

Technical difficulties: The 'Bar Code 1 experience 

A. Using the system requires the instructor to learr. about 
its quirks. Some instructors pnay not have the needed 
time available to learn these. 

B. There is no index, so an instructor cannot pick a 
specific sequence and go back to it 

C. The current version lacks a 'troubleshooting' section to explain how 
to cure problems with the technology. 

D. There is also no mention in the documentation that the 
operator must press 'play' on video disc player before 
other commands may be given. 

Preparation time and independent rp$gareh ji me concerns 

A. Some of the directions seemed very wordy, making it seem 
like it will take longer than it should. 

B. Context Connections: 

1. Requires the user to do some research. Many teachers will not 
have time for this. 

2. This feature may not be found by some teachers, giving them the 
illusion that there is no material available to use with short lessons. 

3. If used this material can be a good bridge to the cultural material, 
but only if the teacher has time to pursue it 

Inefficient use of the medium. (In some areasi 

A. The disc viewed language as the culture, rather than using video to show 
students the culture. 

B. Reading Japanese was introduced in the teaching materials, but no 
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video information was used. 

1. In the reading introduction and in the numbers secti ns the char 
acters are given in the teaching materials, but not on the video. 
The characters for the days of the week were given, so it can be 
done. 

2. The video disc is the ultimate resource (short of a trained instruc 
tor) for teaching reading and it isn't being used for that purpose. 

C. There are some items on the disc that need more than a blue screen when 
they are given. (The months of the year, for example.) 

The lang uage appears to be appears to be atypical 

A. The conversation is one way. You don't hear two children talking 
between themselves. 

B. There are very few action verbs. It is all, T like/ or T dislike' 

C. For some speakers, at least initially, it seems like they are in a race to say 
the words. 

Conclusions 

Many of the assumptions made here are for the benefit of the teacher, who 
unlike the children, will have a harder time learning from this medium. Children are 
natural learners of language, and will often learn regardless of teaching approach. Tho 
goal here is to give them an abundant amount of material to learn, in a manner that will 
help students benefit most We hope the suggestions given will do that 
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Videodisc Evaluation 



Two teams evaluated the Japanese Videodisc program, Konichi Wa, produced by the 
Utah Videodisk Consortium (Art Burnah, Larry Gale, et. al). Our role as evaluators 
was to concentrate on evaluating the videodisc itself. The three main evaluation ques- 
tions focus on the mechanics, teaching, interest, and appeal of the videodisc package. 

Evaluation Questions 

The first set ot questions deal with the mechanics of using the videodisc. The questions 
are as follows: 

1. Mechanics: Does the package work? 

a. Are audio and video intelligible? 

b. Is the speaking speed of Japanese students suitable? 

c. Do Bar Codes work (i.e.: accurate and readable)? 

Question lfal: Are the audio and video intelligible? 
Team 1 Comments 

Team 1 had some difficult) 7 with question (la) in that there was frequently back- 
ground noises that interfered with the audio of the students speech recorded. This 
inconsistency in audio levels occurred in places throughout the videodisk. Also, the 
volume fluctuated between lessons and segments throughout the videodisk. In the fol- 
lowing places on the videodisk the audio was out of sync with the video: Lesson 10 - 
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Segment 4 and Lesson 11 - Segment 5. We feel the images shown are generally of 
good profession quality. The images are sharp and maintain adequate color integrity 
and resolution. The subject matter used appears to be appropriate for the necessary 
instruction. The camera work is professional, although composition and angles of 
some footage are static. Overall the lighting is adequate. However, we noticed that 
flickering occurs in the following full motion video segments: Lesson 7 - Segment 4, 
lesson 10 - Segment 7, Lesson 29 - Segment 2, and Lesson 32 - Segment 5. 

Team 2 Comments 

The audio and video are of high quality and intelligible. The video is done in 
natural settings with colorful backgrounds, and there are a wide variety of settings so 
the learner is constantly seeing new backgrounds even though the content is often 
repetitive, e.g., the student introductions. 

In addition, the settings often complement the content, e.g., on the tennis court 
or with a ball, and this provides valuable contextual and incidental learning. 

The audio appears natural. While the students were clearly coached in what to 
say, they spoke more or less as they would in normal introduction situations. 

Question 1(b): Is the speaking speed of Ja panese students suitable? 
Team 1 Comments 

If the object was for us to follow and understand what the students were saying 
clearly we had difficulty. We felt the students spoke too fast to be understood by non- 
Japanese speakers. 

Team 2 Comments 

The instruction book states that the Japanese language is spoken in syllables. 
However, it was hard to detect syllables, and frequently even words. Consequently, 
the speaking could have been slower and more distinct in the early lessons. While this 
would be somewhat artificial, it would be appropriate in the early lessons where stu- 
dents are learning technique as well as content. 

Question He): Do bar codes work? 
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Team 1 Comments 

We found the bar codes work very well. We experienced some difficulty in the 
beginning, but after we experimented with the Bar Code reader and adapted our 
research to the use of the reader we had no further problems. The following matrices (I 
and II) identifies what we were able to accomplish: 



I. Accuracy and Readability of Bar Codes 



Lesson 


Segment 


Reading 


Tries bef< 


7 


1 


left-to-right 


4 


7 


2 


left-to-right 


2 


7 


3 


left-to-right 


9 


7 


4 


left-to-right 


4 


7 


5 


left-to-right 


2 


7 


6 


left-to-right 


2 


7 


7 


left-to-right 


6 


7 


8 


left-to-right 


14 


7 


9 


left-to-right 


3 


7 


10 


left-to-right 


6 


7 


11 


left-to-right 


2 

Total 54 



II. Accuracy and Readability of Bar Codes 



Lesson Segment Reading Tries before correct 

7 1 right-to-left 3 

7 2 right-to-left 1 

7 3 right-to-left 1 

7 4 right-to-left 1 

7 5 right-to-left 2 

7 6 right-to-left 3 

7 7 right-to-left 1 

7 8 right-to-left 1 

7 9 right-to-left 1 
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7 
7 



10 
11 



right-to-left 
right-to-left 



1 

Total 16 



1 



The total number of trie', to read the left-to-right portion of matrix (I) totaled 
fifty- four as compared to the right-to-left portion of matrix (E) which totaled sixteen. 

Once we knew how to work the Bar Codes with the reader our task to read and 
evaluate the Bar Codes was accomplished without difficulty. Our evaluation of the Bar 
Codes at the present time is that they are readable and very accurate. We did not expe- 
rience a problem with the accuracy (except with the problems we encountered with les- 
son 24 - see inconsistencies section) of going to the correct area or. the videodisk using 
the Bar Codes and the reader. 

Team 2 Comments 
Background 

The purpose of the bar code reader (reader) and the bar codes (codes) printed in 
the manual is to provide the teacher an easy and efficient way to move between the 
instructional segments on the disc. The easy and accurate functioning of the reader and 
codes is the critical link in actual classroom use of the disc. If this works, the instruc- 
tion flows well. If this fails, the instruction fails and teachers quickly abandon using the 
product (or, as seen with Habiar et Parler, improvise by using the videodisc remote 
control unit). 



To test reader/ code function and determine if there is a problem, and further, to 
isolate any problem, the evaluators conducted four consecutive tests. The model of the 
bar-code reader was Pioneer UC-V104BC. The four tests and the purpose for doing 
them is listed below. Note: it should be noted that the evaluator conducting the tests is 
an experienced videodisc user and has successfully used the exact model of bar code 
reader in the past. In addition to this experience, the evaluator practiced for ten min- 
utes with the reader before proceeding with the trials. 

Test 1: Perform 100 trials with the reader and the Pioneer "Command Codes" 



Procedure 



that are sold with every bar code reader. The procedure used was to 
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I 
I 



| 



i 
i 
1 



1 
i 
1 
I 



scan the bar code and listen for the "beep" that indicates a successful 
scan. The purpose for this test was to see if the reader is functioning 
properly. It was assumed that the Pioneer bar codes were reliab 1 ^ 
Test 2: Perform 20 trials with the reader, the Pioneer Command Codes, .d the 
videodisc player using the Japanese videodisc. The procedure used was 
to: 

•scan the bar codes for various functions like step forward, step 
back, play, pause, etc. using as many scans as was needed to 
get a successful read; 

•transmit tne code to the player; 

•observe and determine if the proper command was followed by 
the videodisc player. The purpose for this test was to see if the 
reader, codes and videodisc function accurately. 

Test 3: Perform 100 trials with the reader and the Japanese videodisc bar codes 
provided in the manual. The procedure used was to scan the bar code 
and listen for the "beep" that indicates a successful scan. The purpose for 
this test was to see if the bar codes provided with the Japanese videodisc 
are reliable. 

Test 4: Perform 20 trials with the reader, the Japanese videodisc codes, and the 
videodisc player using the Japanese videodisc. The procedure used was 
to: 

•scan the bar codes for various instructional segments selected 
randomly throughout the manual using as many scans as needed 
to get a successful read; 

•transmit the code to the player; 

•observe and determine if the proper segment was played by the 
videodisc player. 

The purpose for this test was to see if the reader, codes and videodisc function accu- 
rately. * 
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Results 



The results from each test and the conclusion(s) drawn from each test are listed below: 

Test 1: 94 out of 100 trials were successful "reads." This is a successful 

"read rate" of 94 percent. The conclusion is that the bar code reader func- 
tions adequately. 

Test 2: 20 out of 20 trials were accurate. In other words, the disc performed the 
function scanned. The conclusion is that the scanner and Pioneer codes 
function accurately. 

Test 3: 92 out of 100 trials were successful "reads." This is a successful "read 
rate" of 38 percent. The conclusion is that the bar codes provided with 
the Japanese videodisc are adequate. 

Test 4: Twenty (20) out of 20 trials were accurate. In other word, the disc went 
to the proper instructional segment The conclusion is that once a suc- 
cessful scan is obtained, the system functions accurately. 

Conclusions and Recommendations 
Finding One: 

As can be seen from the results, there does not seem to be a problem with the 
reader/code system. The level of "successful reads" is judged adequate for classroom 
instruction. Also, once a successful scan is obtained, it seems the bar code and disc 
function accurately. 



(Note: A caution is needed concerning this finding. The accuracy trials 
used only 20 bar codes selected randomly from the manual. This is judged to be an 
adequate sample for the purpose of this evaluation. However, we heard reports from 
members of other teams that some of the codes were mismatched. Because of these 
reports we recommend that each bar code be checked for accuracy.) 

Finding Two: 

From our evaluation, it is concluded that the reader/code system functions ade- 
quately. If there are reported problems, it may be due to the training of the teachers in 
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using the reader. Because of the central role the reader/ codes play in the overall utility 
of the program, failure to provide adequate training for teachers will be a major obsta- 
cle to dissemination. 

(Note: Another caution is needed here. When our evaluation team first checked 
the reader/ code system, we got extremely low reading rates — in the range of 38 per- 
cent. However, in further testing, we found this to be due to low batteries in the reader, 
Once these were replaced the accuracy rates went to acceptable levels. It seems that if 
there are reports of trouble from the field with the reader, batteries may frequently be 
the cause.) 

Teaching Materials 

The second set of questions dealt with the teaching materials used with the 
videodisc. The questions are as follows: 

1. Teaching: Can it teach students? 



a. 



b. 



e. 



f. 



c. 



What specifically do students learn? 

Are the Bar Code sequences appropriate for teacher 

use (specific, yet flexible enough for teacher ease)? 

Do the icons convey the intended meaning? 

Are the lessons appropriate for 45-50 minute 

sessions (especially for middle school)? 

Is there sufficient cultural material? 

Are teacher explanations quick and easy to use, and 

do teachers actually use them? How much teacher 

time is involved? 



Although our group was not assigned to evaluate the teaching materials, there 
are some questions pertaining to the teaching materials which can be parrtially 
answered in conjunction with evaluating the videodisc. The observations of the two 
teams and modification suggestions are stated below. 
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Question 2: Can it teach students? 

We found the teaching portion of the videodisc package to work quite well, for 
the most part. We had some difficulty with certain areas of this portion of the 
videodisc. 

Can it (videodisc) teach students? Our answer to this question is somewhat of a 
guess, at this time, but we feel that it will be able to accomplish this end if some change 
are incorporated in the teaching materials. The following are a number of inconsisten- 
cies that were noticed which require changes in the teacher guidebook: 

1. Lesson 1 reads Segments 1-7 in the strategy section, but there are 1-9 
Segments shown in the bar code section. 

2. Lesson 9 reads Segments 1-5 in the strategy section, but there are 1-6 
Segments shown in the bar code section. 

3. Lesson 12 reads Segments 1-5 in the strategy section, but there are 1-4 
Segments shown in the bar code section. 

4. Lesson 19 reads Segments 1-12 in the strategy section, but there are 
1-13 Segments shown in the bar code section. 

5. Lesson 25 reads Segments 1-8 in the strategy section, but there are 





1-10 Segments shown in the bar code section. 






6. 


Lesson 34 reads Segments 1-11 in the strategy 
1-12 Segments shown in the bar code section. 


section, but there 


are 


7. 


Lesson 44 reads Segments 1-14 in the strategy 
1-15 Segments shown in the bar code section. 


section, but there 


are 


8. 


Lesson 46 reads Segments 1-12 in the strategy 
1-13 Segments shown in the bar code section. 


section, but there 


are 


9. 


Lesson 51 reads Segments 1-14 in the strategy 
1-13 Segments shown in the bar code section. 


section, but there 


are 


10. 


Lesson 56 reads Segments 1-14 in the strategy 
1-15 Segments shown in the bar code section. 


section, but there 


are 


11. 


Lesson 57 reads Segments 1-13 in the strategy 
1-14 Segments shown in the bar code section. 


section, but there 


are 
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In other parts of the videodisc we discovered some inconsistencies. Please note 
the following inconsistencies: 

Captions were a problem (on the Bar Codes) throughout the guidebook. Some 
examples can be found in Lesson 11 (no notification if audio or video is being used), 
Lesson 12 (no video present and no note defining that), Lesson 22 (no captions on the 
Bar Codes), and Lesson 29 (two segments with incorrect captions). 

The color lesson of Lesson 24 was very confusing. Colors shown on the monitor 
screen did not match colors called out in the guidebook. We feel a complete review of 
this section is necessary to correct this problem. 

We were impressed with the video that was used in the first 47 lessons, but we 
became saddened when we viewed the animation that was put into Lessons 48, 49, and 
50. It was disappointing as animation and was a stark contrast to the excellent video 
shown in the other lessons. 

Why were the lessons set up in such a way that we continually had to turn the 
videodisc from side one to side two? As an example: We thought Lesson 29 and beyond 
were on side two of the videodisc. We were surprised to find that Lessons 30 through 
35 were on side one; Lessons 36 through 53 were on side two; but Lessons 54 and 55 
were on side one; whereas Lessons 56 and 57 are on side two. From a user standpoint, 
these arbitrary changes make the product somewhat less user friendly than would 
seem to be necessary. 

Question 2(al What specifically do children learn? 
Team 2 Comments 

The lesson objectives are clear. They are to learn how to introduce oneself with 
name, age, hobbies, and what town they are from. Other objectives include learning 
vocabulary such as fruits, different clothing items, transportation modes, parts of Japan 
(cities) and phrases as time location and weather. 

Question 2(b\: Are th e bar code sequences appropriate for teacher use? 
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Team 1 Comments 

As for question (2b) we can give an affirmative answer. The bar codes seem to be 
accurate and readable. 
Team 2 Comments 

Are the bar codes sequences appropriate for teacher use? (specific, yet flexible 
enough for teacher ease?) Bar codes sequencing seems relatively unimportant because 
operation of the scanner and box are not affected by sequence. However, there may be 
some value to the teacher in anticipating the general code numbers for manual entry. 

Question He): Do icons convey the intended meaning? . 

Team 2 Comments 

As the instructor uses the disc, the icons will become more familiar. At first, 
some may not understand the icons 1 intended meanings. 

Question 2(d): Are the lessons appropriate for 45-50 minute sessions (especially for 
middle school? 

Team 2 Comments 

Yes. The lessons are appropriate for 45-50 minutes sessions. Each of the lessons 
are broken down into five sections composed of introduction, modeling, guided prac- 
tice, individual practice, and culture connection. Each sections includes one or more 
activities. The time each of the activities takes varies from 2 to 10 minutes depending 
on the nature of the activities. 

For example, in the lesson to teach the name of the colors, the teacher will first 
discuss with the students some of the feelings and emotions that people generally 
describe with colors and then inform the students that they are going to learn colors. 

Then, the students will cut out the colored construction paper to make a set of 
flash cards, and use the flash cards, and use the flash cards for the color name practice, 
etc.. 

The activities of each lessons are long enough to keep the students occupied for 
45-50 minutes, but whether these learning activities can attract the students' attention 
for so long need to be found out through observation. 

Question 2(e): Is ther e sufficient cultural material? 
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Team 1 Comments 

We found the videodisc to be significantly cultural as noted in question (2e). The 
videodisc covered the major areas of Japanese life (i.e.: historical, home, work, school, 
and recreation). Designers of Kanichi Wa did an excellent job in to conveying cultural 
material. 

Team 2 Comments 

The cultural shots and content in the video disc are very good and are appropri- 
ate throughout the disk. There really isn't much more to say about this. It is either 
good or not. 

One of the evaluators had learned the language as an LDS missionary nearly a 
decade before and had been extensively involved with the culture since then. It seemed 
to him that it would be more effective to include more of the culture earlier on in the 
lesson. 

Question 2(f\: Are the teacher explanations quick and easy to use, and do teachers 
actually u se them? How much tparhPT fli pe is involved? 

Team 1 Comments 

The first part of the question (the teacher explanations are quick and easy) 
receives a resounding yes. However, we feel the teacher will have to take an active role 
as the facilitator of this material for it to succeed in the classroom (although at this time 
we don't know how much of the material the teacher will actually use). Just as the stu- 
dents in the classroom will learn Japanese, so also will the teacher learn the Japanese 
language. This learning will require a substantial amount of time by the teacher. 

Team 2 Comments 

Yes. The teacher explanations are easy to understand and use. We spent about 
15 minutes to reading the explanation and were able to start the lessons. 

It is our opinion that if a teacher spends about half-an-hour to read through the 
explanation and another half-an-hour to prepare the activities and watch the lesson 
segment, he or she will be well prepared to teach the lesson. 

A shortcoming in the lesson book which we noticed is that the book does not 
have a table of contents. It is hard to find specific lessons or contents. 



Konichi Wa Evaluation Report Page 35 



Utah State Evaluation Team 



Interest and Appeal of Program 

The third set of questions deal with the interest and appeal of the video disc program. 
The questions are as follows: 

3. Does the program have interest and appeal? 

a. How do students react initially and after continued 
exposure? 

b. Which parts of the instruction package are most attractive 
and which are least attractive? 

Although our group was not assigned to evaluate the interest and appeal of the 
videodisc program, there are some questions pertaining to its interest and appeal which 
can receive a partial answer in conjunction with evaluating the videodisc. The observa- 
tions of the two teams and suggestions for modification are stated below. 

Question 3: Does the program have interest and appeal? 

Team 1 Comments 

Our overall feeling about the videodisc is that it is well done and will be an asset 
to the teacher in the foreign language class. We feel that there will be high interest and 
appeal ratings from the students, if their attitude of the program is similar to ours. 

Question 3( a): Which parts of th e instruction package are most at tractive and which 
are least attractive? 

Team 1 Comments 

We feel the scenes of Japan are a highlight as well as the candid shots of the chil- 
dren in their various activities. 

The constant turning of the videodisc because of the lessons being out of 
sequence hurt the overall appearance of the quality of this product We found the poor 
animation in place of the excellent video in several of the later lessons hurt the overall 
professional appearance of the videodisc package. 
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Team 2 Comments 

As for the lesson objectives, they are adequate. However, we do not see how 
having a dozen or more people introducing themselves is anv more of a benefit to the 
learners. There are times when hearing seven people in a row became tiresome. It is 
good that the bar codes were done so that only one person would be shown introduc- 
ing themselves. 

We also feel that the disc needs to be re-arranged as far as the lessons are con- 
cerned. The material about Japan itself is left towards the end. We feel that to spark 
the interest of a new learner to a subject it would be good to give a introduction to the 
subject in this case showing parts of Japan and a bit of the history of the country. 
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Cultural Supplement Materials 



Global 

Konichi Wa 
Introduction 

This learning project has 45 lessons, with each lesson separated into three parts: 
Objectives, Materials, and Background Information. These sections aie followed by 
supplemental enrichment materials such as pictures and informational tid-bits. 

The 45 topics focus very well on Japanese culture. One evaluator involved with this 
analysis, Hiroshi Nakayama, is native Japanese, and feels qualified to give his opinion 
on this subject. The Objectives, Materials, and Background Information of each section 
provide good focus points for each topic; however, we feel that one page of text is 
insufficient Additional materials are needed that the teacher can draw from. 

Format 

The numbering throughout the documents needs consistency. The text font was dis- 
tracting and occassionally hard to read. Elements of page design were at times lacking. 
An overall feel of consistency was missing from the materials which gave them the 
impression that they were thrown together. Just attending to simple elements of head- 
ers, fool rr>, consistent page formatting etc. could provide additional professionalism to 
the document. The use of an appendix and index would also aid the teacher in using 
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the materials more efficiently. This appendix could hold all of the Background pieces 
rather than inserting them at the end of each lesson without page numbering. 
Lastly, we suggest the of the materials could be improved by reorganizing the the table 
of content to replace the overwhelmingly lengthy one now being used. For example: 

Section 1: History and Background of Japan 

Section 2: Cultural and Customs 

Section 3: Social Life Style 

Section 4: Education and Arts 

Section 5: Others 

Later in the analysis we suggest a template of some sort that could be added to the 
front of the material that would aid the teacher in preparation and presentation of 
lessons. 



Analysis Organization 

The following evaluation is broken into 6 sections: 

1. Global Analysis 

2. Content 

3. Instructional Design 

4. Record Keeping 

5. Ease of Use 

6. Conclusions 



Global Evaluation Matrix 





Superior 


Very Good 


Good 


Fair 


Poor 


Content 




X 








Instructional 
Design 




X 








Record Keeping 










X 


Ease of Use 






X 
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I. Package Contents 

1 . Resource Video Disc 

2. Support Manuals and Lesson Plans 

3. Cultural Materials 

II. Hardware Requirements 

1 . Video Disc Player 

2. Bar code wand 

3. Display Monitor 



Content 



I 
1 
I 



Content Evaluation Matrix 


Excellent 


Satisfactory 


Unsatisfactory 


Not applicable 


Objectives are fully and clearly defined 


X 








Target audience is clearly defined 




X 






Prerequisite skills are clearly defined 






X 




Content is consistent with objectives 


X 








Vocabulary level is appropriate for subject area 
and learner level 


X 









I 
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Content 



Content Evaluation Matrix 


Excellent 


Satisfactory 


Unsatisfactory 


Not applicable 


Objectives are fully and clearly defined 


X 








Target audience is clearly defined 




X 






Prerequisite skills are clearly defined 






X 




Content is consistent with objectives 


X 








Vocabulary level is appropriate for subject area 
and learner level 


X 









The cultural materials help students learn about another culture by familiarizing 
them with their own. This method of comparison is a good learning strategy. 
Familiarizing the children with their own culture and then examining parallel elements 
in another culture will only enlighten the children's understanding of the concept of 
other existing cultures. The attempt of the cultural materials to teach children aware- 
ness and respect for other cultures is admirable. However, that attempt cannot solely 
be accomplished by the cultural materials; the teacher will have to play a large role to 
succeed with this objective. We agree that the content of the cultural materials is clearly 
presented and outlined in a understandable fashion. The content does accomplish its 
listed objectives for the individual chapters. We do note that there were no objectives 
listed for the contents as a whole. The content is written in a format that we feel is 
understandable to the children at the elementary level 

We are concerned that the cultural materials are not linked to the videodisc. We 
were under the impression that the videodisc and the cultural materials were one pack- 
age since they are packaged together, and if they are one package, there should be some 
link, even if only instructions that tell what part of the cultural materials can be found 
on what part of the disc. We are aware that the cultural materials were produced sepa- 
rately by Clark County (Nevada) School District. Knowing this, it is still important to 
point out the importance of linking the two. We are dso concerned that there is no 
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introduction to the cultural materials, we feel that an assumption should not be taken 
that everyone will be familiar with its contents. Also along the same lines, there is no 
instruction to use the manual. Another assumption that we feel is not warranted: there 
are no prerequisites assigned, and if there are no prerequisites required then there is 
nothing stating that fact. The adequate use of the cultural materials seemed to us to be 
dependent on the teacher. If the justification for this is that it is because the target popu- 
lation is elementary school children, then anyone using the cultural materials would 
not be able to recognize that fact because there is no target population listed with the 
cultural materials. 

The concept of the cultural materials is valuable and with Asian cultures almost 
always necessary. Some of the teaching strategies used in these materials were effec- 
tive. The structures of the individual chapters were organized well within themselves 
but lacked such organization as a whole. 

Instructional Design 



r 

ID Evaluation Matrix 


Excellent 


Satisfactory 


Unsatisfactory 


Not applica- 
ble 


Learner controls rate and sequence of instruction 








X 


Program can be used independently 




X 






Learner interacts only with appropriate segments 


X 








Program utilizes a variety of display and response 
modes 








X 


Program minimizes necessary typing 








X 


Program handles a wide range of student responses 


X 








New material is presented in context and is related 
to previous material 




X 






Summaries and reviews are provided, important 
concepts are restated 


X 








Program can be adjusted by user for local needs 








X 


Appropriate use of graphics/color/sound 




X 




X 
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ID Evaluation Matrix cont 


Excellent 


Satisfactory 


Unsatisfactory 


Not applica- 
ble 


Feedback is useful and appropriate 








X 


Instruction is active rather than passive 




X 






Learner expectancies are established 




X 






Program has a consistent display rate 








X 


Displays are clear, understandable and effective 








X 



In evaluating the instructional design for the cultural materials that are used in 
the Japanese video disk program some definite strengths and weaknesses were deter- 
mined. It was determined that the teacher could use the materials independently from 
the video disk and that the units provide a variety of required responses. The sections 
are written in such a way that teachers can integrate them into their own teaching 
method. The materials also seem to be developed to encourage students to interact in 
their learning experience through discussions, mini-lessons, and presentations. The 
flexibility given to teacher on how to integrate the cultural material is good, but when 
delivering a new program some direction on how to proceed is required. With the 
Japanese Cultural material there is no attempt at giving strategies or explanations to the 
teacher on how to best present the units or how evaluate the students progress. This is 
a weakness that needs to be corrected. 

Each section dealing with the history of Japan builds upon the prior historical 
units, and each of these sections give a brief yet clear and understandable overview of 
historical events that have occurred within Japan. This portion of the program, howev- 
er, appears to be the only part of the cultural material that builds upon the previous 
units. The others units are sectionalized or isolated. They touch on different areas of 
the Japanese culture, but there is no attempt to tie them together. 

Overall the Japanese Cultural materials provide a good learning experience and 
with a few modifications the learner will have a great experience in learning about the 
culture of Japan. 
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Record Keeping & Management 



Record Keeping & Management 
Matrix 


Excellent 


Satisfactory 


isatisfactory 


)t applicable 








Z 


Program keeps records of student response 








X 


Program keeps on going student records 






X 




Program includes diagnostic/evaluative testing 






X 




Program generates further assignments 




X 






Program graphically depicts students progress 








X 


Program provides statistical information on 
student progress 








X 


Program allows printout and screen display of 
of student records 








X 



There is no system that we could find that would help or facilitate record keep- 
ing or student tracking. Although this criterion might apply more to computer based 
training than to support materials/ there still needs to be some system of evaluating 
how well a student is learning. Many teachers have b n making their own tests for a 
variety of subjects, so maybe this is not a major concern, but it would seem that an inte- 
grated assessment system that would tie into the already well designed materials, 
would provide the finishing touch to a quality project 

A language program seems to be dependent on the cultural aspects of the native 
people and country. In the case of Japanese, we would think this dependency only 
becomes magnified because of the extent that tradition walks hand in hand with mod- 
ernism. There are so many parts of Japanese culture that affect both the language usage 
and non-verbal communication. To have no assessment of the ability of the students 
leaning this very pertinent information seems to be a wrongful exclusion. 

The program does suggest other activities that could be done for assignments 
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with in some lesson plans. This aspect encourages some generalization and transfer of 
many vital components. In watching these extra activities, the teacher could watch and 
evaluate how well the student is learning and grasping the subtleties of the culture. 



Ease of Use 



Ease of Use Matrix 


Excellent 


Satisfactory 


Unsatisfactor 

y 


Not applica- 
ble 


Support materials provided are comprehensive 
and effective 


X 








Program is reliable in normal use 








X 


"Help" procedures are available 






X 




Program can be exited by student when 
appropriate 








X 



The designers of the cultural materials have gone to great lengths to provide 
easy-to-understand enhancements to video disk exercises. The designers of the materi- 
als have done a superb job of providing materials that any teacher could pick up and 
use in his/her classroom. The students will gain much from what is contained inside. 
They will learn of Japanese history, the major cultural differences, the many hardships 
as well as benefits of living in a country such as Japan. The students will come to 
understand the great traditions that are still respected and followed by many of the 
Japanese people. The materials are very extensive and inclusive of inform* tion neces- 
sary to understand and appreciate the Japanese culture. 

Our only suggestion is that there be more front end organization materials that 
will inform the teacher about what to be prepared for before proceeding. The materials 
lack clear guidelines and objectives overall. They list objectives for each lesson, but 
there is no formal "mission" statement that refers to the materials as a whole. The 
advantages of such a system take away the overwhelming feeling of taking on a project 
such as this, and only add to the excitement of teachers looking forward to using the 



Page 46 Konichi Wa Evaluation Report 



Appendix C 



materials. Such an organizing scheme would also provide the teacher with realistic 
expectations of time and tool use. It would help them arrange their preparation time 
better and allow them to be better prepared. Though there were activities introduced 
and questions at the end of each section, there was not any formal suggestion how 
these materials should be used. Maybe that information is in the Konichi Wa manual 
or tucked somewhere in the cultural materials. 

The information seems to be both reliable and comprehensive. It is structured in 
such a way as to encourage teachers to use their own techniques in the classroom set- 
ting. It also provides for a varied response range. Over all we felt that the Japanese 
Cultural Materials were extensive and should provide adequate support for the teach- 
ers using the videodisc. The package contains extensive lesson plans that provide the 
teacher with the additional materials needed to enrich a foreign language learning set- 
ting. 

There are a few short comings of the materials that are minor but worth men- 
tioning. Because of the amount of material provided, they may be overwhelming to a 
new teacher who is already apprehensive about teaching a language where he or she 
lacks experience. A short guide that maps out exactly how to use the materials in the 
front of the materials would be helpful. There is an introduction, and there are para- 
graphs that list the objectives and purpose of the materials and there is even a short 
message that informs the teacher how to use the information with the videodisc, but 
there is no formal plan that lists the amount of preparation time that will be required 
per lesson, or what tools would help enhance the lessons or variations that could be 
added to the lesson (or even if that was possible). It just needs some type of organiza- 
tional template that would help inform the teacher about what is about to be intro- 
duced and what the requirements are going to be to return that information to the stu- 
dent 

Once the organization is clear the manual is very easy to use. The manual lack 
any kind of assessment instruments that the teacher could use to access students reten- 
tion of material. Obviously lacking was any kind of student tracking. Even if the stu- 
dent is not going to be assessed, the teacher must still keep track of the students 
progress through the lessons. If teachers feel these things are needed they will have to 
spend their own time developing them. 
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One of the most positive aspects that we found in the materials was the 
approach of helping the student learn about another culture by helping him/her 
become more aware of his/her own, The materials try to teach awareness and respect 
of other cultures. The material is written using concepts that are easily grasped by 
young students, so less is required of the teacher to adjust them to their level. This 
point of correct concept placement is very interesting because the materials never spell 
a target population, but they do address a certain age group. 

Even with the quality of teacher in the present educational system, we believe 
the materials leave too much up to the teachers. The lesson plan contains a lot of infor- 
mation, but there are not many suggestions on how to present that information to the 
students. However, these cons can also be pros in some people's eyes. Lack of restric- 
tions breeds freedom of expression and creativity. We just don't know how many 
teacherswill take the initiative needed to motivate the students to learn. The materials 
do encourage teachers to ask their students questions to build interaction. Such loose- 
ness in structure may also provide the teacher with the flexibility to respond to varied 
responses. 

Conclusion 

We do feel the program as a whole is very valuable and produces effective 
results. The cultural materials compliment the videodisk program. They also provide 
knowledge about the people of Japan that will probably not be forgotten. It will be the 
people and the foreign scenes that will be remembered long after the students memory 
of the language has faded away. The cultural materials provide information that helps 
students respect the Japanese people. 

It has long been known that people mock whai they do not understand. We 
believe it is possible to learn a language and still have no respect or little appreciation 
for the people who speak it. Thanks to the cultural materials, students will have a bet- 
ter understanding of the people of Japan. This includes their dress, values, history, 
games, government, and other aspects identified in the cultural materials. 

During our on-site evaluation in Las Vegas March 3rd and 4th, middle school 
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teacher Patricia Arroyo was interviewed in Las Vegas. She said that there have been 
"several 77 instances where students have returned from their first couple of Japanese 
classes with reports of unhappy parental reactions. These parents wanted their stu- 
dents taken out of the language class because they did not want their child learning 
about Japanese people. Patricia was asked if these frustrations stemmed from cultural 
biases. She quickly replied, "Yes." 

Patricia stated that, when she first started showing the videodisk, students 
would make fun of the Japanese children identified on the screen. She said, "Children 
can be very mean," and she added, "The videodisk program (cultural materials) has 
provided understanding- and now the children don't make fun of the people anymore." 

Also while talking with the children in Las Vegas at Orr Middle school, students 
identified their favorite Japanese characters. Across the board, Mami was selected as 
the favorite. All the students agreed that she is the loudest, clearest and easiest to 
understand. And one student added, " She is pretty." 
Cultural Supplement Materials Evaluation Team 

Thanks to the cultural materials, children at Orr have not only stopped discrediting the 
people, but they also relate to them and appreciate them for what they represent as 
members of Japanese culture. 
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Las Vegas Observation at 
Gibson School 

Classroom Observations 

The students who attended the Gibson middle school were seventh grade stu- 
dents who choose either to take the Japanese course or a computer science class. The 
course was in its sixth week of instruction for the current term. The teacher, Mr. West, 
was teaching the course for his third time. He began teaching with the Konichi Wa 
videodisk in September of 1993. Until that time Mr. West had no experience with the 
Japanese language, but he taught Spanish and other language classes at Gibson for 17 
years. There were 10 girls and 12 boys attending the first class and 9 girls and 11 boys 
attending the second class. The ethnic background of the students was mixed. 

Attitudes 

The class as a whole was energetic about learning Japanese. Most students were 
enthusiastic about responding to teacher questions and did not shy away from speak- 
ing in front of the class. Every time the teacher asked questions at least five hands 
went up (and they weren't always the same people). Each student possessed confi- 
dence about the knowledge they had developed. The level of learning seemed consis- 
tent throughout. The students respondent to the videodisk instruction. We noticed 
that the students paid attention to the video monitor when sections were played. 

Accomplishment and Language Skills 

We were impressed with dialect and understanding of content. They were able 
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to give their names, where they live, what they like, and what they do not like. Some 
student even were able to identify any number chosen by the teacher under 100. They 
also were able to identity in Japanese different types of wild and domestic animals. 
They responded to questions asked in Japanese by giving answers in Japanese. 

Learning strategies 

Both classes we observed were identical in the teaching strategies. They began 
by asking the student to recall prior learning through oral repetition. After a section of 
practice cultural materials were presented. Notes were being taken by the students on 
what the teacher was telling them. Then the cultural knowledge was repeated through 
of the videodisk. The sections were played several times. The students continued to 
fill-out their sheets. 

Focus Groups 

Group One 

Haw long have you studied Japanese? 
"Six weeks." 

Haw do you like Konichi Wa? 

"I could not hear everything they said. " 
"It was hard at first" 

"It was hard when they talked really fast" 
"My Mom thinks I should take it next year. " 
"Sumo wrestling is really cool." 

What Don't you like? 

"Tests. We have open note tests. " 
"Lots of information and lots of paper." 
"I just wrote it ail on. one paper." 

Who don't you like? 
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"I don't like Tacoma; his voice is really low and he talks fast/' 
"He was hard to understand/' 

Wliat can you say? 

" All of the kids could introduce themselves, state what they like and don't like, 
and where they were from/' 

Is it scary to get up in front of class ? 

"No not really. Girls are more shy to get up in class/' 

Hard to learn Japanese? 

"No it is easier then we thought it wouid be." 

Cultural things? 

"If went to Japan, we could ask for directions because we know the basics." 
"What to get and what not to get" 

"We would know what to do if someone invited us to their house. " 
"We would know to take off our shoes, bring a gift, and that guest sits in the 
most honored place." 

What would you want to do in Japan ? 
"Watch Sumo wrestlers." 
"Go to the mountains." 
"Go to a good restaurant" 
"Eat a snake." 

How do you learn Japanese best? 

"By watching the video." 
"Watching TV was more interesting." 
"I like the worksheets." 

"There was a problem with the worksheet papers." 

"You can drop things and they get messy. We want it organized " 

How can you learn Japanese better? 
"No ideas." 
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First week in class? 

"I was scared/' 

"Couldn't understand people. " 
"I didn't think I could do it/' 

"I like it more now, at first I didn't think I could do it" 

Is it a hard language to learn ? 

"Not when you get down to it" 

"It getseasier once you break it down." 

"When you think about it they aren't talking that fast" 

"Learning this will help me in high school." 

WJio would take Japanese in high school? 

"I would if they taught it like he does." 
" I would like to take a Spanish videodisk class." 

(One person said maybe while the rest of the five said yes All said they would want to go 
to Japan and teach them English. Kids wanted to know what other discs are available? They 
want Spanish, but would take Japanese over Spanish). 

Is this your favorite class? 

All of the kids agreed they learn the most in this class. The integral visual aids help the 
students learn faster. The students all agreed that this was their favorite class, except PE for one 
kid who said, 

" I already have a billion things to do." 
The others: 

"We learn a lot more in this class. It goes faster when you are having fun. " 
"It is my easiest class because of the videodisc. 

"They teach you something then you can understand the characters in the 

video." 

"It is fun." 

Group Two 

Haw do you like Konichi Wa? 
"It is cool." 
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Is this your favorite class? 

"Yeah, because it if fun/' 

"We are doing something besides school work/' 
"We get to learn a language. " 
"Second behind PE. " 

Favorite character? 

"Basketball character. " 

Counting? 

"One boy counted to 30." (Most could do some counting.) 

Observation 

We began our business by meeting Elena in the hotel lobby. We proceeded to 
Orr school without much interaction. Jeff and Darren began work at Orr while we ate 
breakfast and spoke with Elena. The conversation was interesting and intellectually 
challenging. The subject matter of our conversation covered the introduction and 
implementation of the videodisk. Elena was positive about the videodisk thus far. 
From her standpoint no glaring problems have arisen. It was interesting to hear Elena 
bring up the lack of training fro teachers in the implementation of technology. She also 
noted the lack of instructional technologist in education in the district, but she conclud- 
ed there are not any job openings for them. She was interested in the theories of learn- 
ing and wanted to know more about the genre of literature supporting instructional 
technology. She talked about the raising awareness of cultural attitudes as the children 
learned a foreign languages. Overall, our visit was enlightening and pleasant 

Gibson school students came from a lower-middle sociological group. It is very 
multi-cultural: Hispanic, Black, White, American Indian. No group dominated yet the 
majority of instructors were Caucasian. The Principal is a black female around 50 years 
old. We interviewed her briefly. She had only heard very positive comments. The 
only negative comment reported was that there wasn't a follow-up program so stu 
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dents could continue their study. She believes that the instructor was doing a good job. 
Later comments she received, verified her impressions. 

Mr. West has taught at Gibson for 17 years. He is a reading and foreign lan- 
guage teacher. His language experience was in Spanish and Russian. The kids in the 
first class were more hyper (energetic, activated) than the afternoon session. Mr. West 
informed us that some of the kids in the earlier class were from special education. One 
girl was disruptive to the point where she was sent to the office. Later in the dav we 
discovered that her brother had been killed in a drive-by shooting two days earlier. 
This dramatic incident at home is indicative of the kind of issues which the teacher was 
facing. The atmosphere for teaching was not exactly ideal. With all these distractions, 
teaching still took place. 

Mr. West began by having randomly selected children get up in front of the class 
and in Japanese state their name, what city they lived in, what they liked , and what 
they disliked (which was invariably school or homework. The children were a bit ner- 
vous in front of the camera, but they performed w r ell. Then, he asked questions in 
Japanese and the children responded in Japanese. (We don't know what the subject 
matter was.) The children seemed quite adequate in their responses. They were also 
eager to participate and many times Mr. West had to regulate response. Later Mr. West 
commented they were not just acting for the camera, but that this has been the pace for 
the course. Following the Japanese questions he played sections of video showing ani- 
mals with Japanese audio; then the video only and ask the children to pronounce the 
words. Afterwards he lectured in English on cultural material and asked the student to 
take notes. When the material was covered, he played a three minute clip of a video 
representation of the cultural material. 

After first period we talked to Mr. West about the program. He really likes the 
courseware materials. He felt that it was a positive experience for the children. A num- 
ber of the students who perforated poorly in other classes have performed well in his. 
Each student entered the course with no understanding of the language. Consequently, 
the class progressed together in their acquisition of knowledge. Some ^luldren felt they 
were not so stupid and could excel in the subject. Many self-esteem issues were dis- 
cussed. After lunch we observed half of the second class and conducted a focus group. 
The structure of presentation for the afternoon class was similar to that observed in the 
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morning. The focus group went very well except for a number of buses driving by 
which made it impossible to hear responses or use portions of the interview for video 
footage. 

We met Elena Steel at 4:00am in her office. Much of the time was spent observ- 
ing and discussing a Spanish video they have been developing. She is excited about the 
Japanese program and our evaluation process. She has no complaints about how the 
system is working. Presently, she is comfortable with all aspects of the Japanese pro- 
gram. 

Day two we returned to Gibson middle school and ran a focus group with the 
first class. Our second focus group happened to draw-out a quiet group of children. 
We observed the first half of the lesson which followed the same structure as the first 
day. We had to pry to get information out of them. After the focus group a discussion 
was held with the entire second class. Some very interesting points were discussed. 
First, all of the students enjoyed the class and stated that it ranks in the top of their list 
of courses taken at Gibson. Secondly, the students were asked if they were nervous 
about taking a Japanese class. All but a few hands went up. A follow-up question 
asked if they would take another foreign language course. Roughly 15 students out of 
the 20 in the class said that they would. We asked they why their attitude has changed 
about foreign language and students replied that it is not that hard. They talked about 
how cool it is to learn another language. Many students noted they could go home and 
impress other with their language skills. The children talked about empowerment 
issues (e.g., getting a job). 

The students like the videodisc. One student commented that it makes a com- 
fortable atmosphere. He said they were accustomed to watching television at home 
and now it is cool to be able to watch in class. Another student commented that she 
liked being able to model the dialect. They could see the person, hear them, and then 
practice speaking. One student even said her mother wanted a copy of Konichi Wa so 
she could see it at home. 

We asked what they did not like about the program. During class with all of 
their class mates around nobody had one negative comment. The entire atmosphere 



Konichi Wa Evaluation Report Page 57 



r|c 56 



Utah State Evaluation Team 



was favorable towards all parts of the course. In the focus group we heard complaints 
about taking notes and the test; but in the class discussion these issues did not arise. 

We had a final interview with Mr. West as we began wrapping up our observa- 
tion. He enjoyed having us observe his course and perceived the students were more 
excited about the course. He wished there were more programs that could take the lan- 
guage farther for the students (beyond his course), but mentioned state regulations that 
required a teacher to certify in the language before they could teach it past the nine 
week course level. 

After the second day of observation the evaluation team met with Elena Steel for 
an exit debriefing. We wanted to know if the teacher selection was unbiased. Our con- 
cern was that both teachers were excellent teachers. We wanted to know if she sent us 
to the best situations for observation. Elena said she has nine school that have the 
adopted the program. Only two have several courses being taught. She choose which 
schools the observation team would visit based on this information. She had visited 
Orr middle schoo,l but had not seen how things were going at Mr. West's class. In the 
end, we felt comfortable with the sample of teachers. 

Conclusions 

The experience developed a much greater understanding of the evaluation 
process for each of us. It was a good experience to take ideas and tactics we learned in 
class and apply them in a real world situation. Regrettably not all students in the class 
could have the same experience. Our situation worked well. Because of the nature of 
our group the students responded to both of us in an amiable manner. 

Our impressions of the videodisk course are all positive. If the objectives are for 
the students to have a positive introduction to a foreign language, then when 15 out of 
20 students say they want to take another foreign language, the results speak for them- 
selves. The students are learning the language and enjoying the media of presentation. 
The only negative comments were not major issues. In the situation we observed the 
program is operating smoothly. 
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Las Vegas Observation at 
Orr Middle School 

This paper identifies our general findings and opinions developed during the on 
sight evaluation of the Konichi Wa Japanese language video disc produced by the Utah 
Videodisc Consortium. For our evaluation we used several assessment tools including 
class observation, student focus groups, teacher interview, and a teacher survey. The 
evaluation took place at William E. Orr Middle School located at 1562 E. Katie, Las 
Vegas, Nevada. 

Teacher Profile 

Ms. Patricia Arroyo has been teaching at Orr Middle School for over 10 years 
now. For the first 10 years she taught English as a second language to students who 
did not speak it as a native language. She said that she has enjoyed reversing the role 
and teaching other languages as the second language this year. Ms Arroyo could be 
considered a very motivated teacher. She was so excited to be teaching Japanese this 
year that during the summer she went to UNLV and took a Japanese class to prepare 
herself. She has been teaching Japanese since the beginning of this school year in 
September. One of the unique things Patricia did to make the class more of an experi- 
ence was to schedule a night at a local authentic Japanese restaurant for any of the stu- 
dents and their parents. 

Classroom Demographics 

The students who attend Orr Middle School represent a diverse set of cultural 
and ethnic backgrounds with no one particular dominant group. Many of the students 
come from lower-middle income families and live in apartments, not homes. The 
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neighborhoods where these children grow up are rough. Waiting for the second class 
period we observed posters on the wall in the hallway depicting highlights and low 
points in the student's lives (one student having had a bad day after being shot) The 
students in Ms. Arroyo's classes were sixth grade, 17 boys and 11 girls in the morning, 
14 boys and 13 girls in the afternoon. The students were completing their sixth week of 
a nine week term during our visit 

Student Attitudes 

The videodisc was very effective at gaining and holding the students attention. 
Without direction from Ms. Arroyo the class quieted down quickly when the videodisc 
came on the screen. As she reviewed the material from the previous day the volunteer 
participation rate was between 75 and 85 percent On several occasions a student asked 
if Ms. Arroyo could play the clip again because they did not get it the first time. 

Accomplishments and Language skills 

Memorization of words and phrases was obvious during the review at the begin- 
ning of each period. Ms. Arroyo would give the students a word or phrase in English 
and ask for a translation. During the focus groups we asked one of the students to tell 
us their favorite phrase in Japanese, then we asked one of the others to interpret for us. 
In each case the interpreter was correct, according to the first student and the others 
agreeing. Character recognition also appeared to be taking place, as Ms. Arroyo drew 
the Kangi characters on the board and asked the students how many this was. 

Learning strategies 

The learning strategies appeared to be effective from the observations of the in 
class reviews. The strategies Were a combination of drill and practice and the use of 
drawing Kangi characters to reinforce the knowledge through another channel. The 
information on the videodisc was used extensively as a model for the students. 

Classroom Observations 
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Classroom Observations, morning class March 3 

On the first day of the evaluation, we spent our time in the 9:00 am class observ- 
ing the instructor using the video disc. She used the disk to teach students the name of 
each day of the week. It seemed, as she turned on the video and turned off the lights, 
that each of the students gave their complete attention to both her and the monitor. At 
the completion of this step, student were able to recite the names of each day collective- 
ly. An individual assessment was not obtained by the instructor at this time. 
She then wrote numbers 1 through 12 on the chalkboard in both Japanese and English 
and had the students collectively and individually vocalized the name of each number 
in Japanese. 

She concluded the class lessons by distributing paper , brushes, and ink for each 
student to try their hands at making their own Japanese Kangi. She used the videodisc 
as a model for students to follow in their artistic endeavors. Students worked slowly 
and carefully to maintain consistency with the images provided on the videodisc. As 
the students worked, she asked several of them to recite the name of each number in 
Japanese. These students appeared to be proficient with little of no coaching. 

Focus Group #1 afternoon class. March 3 

For this days 2:18 class, we randomly selected 6 students to leave the classroom 
to participate in a student focus group. Jeff began this meeting by asking each of the 
students to say a statement in Japanese and then asked another student to interpret 
what the fist had said in English. In every instance, the second student was successful 
at identifying the first students statement. 

Also in this focus group, students identified their favorite Japanese characters. 
Across the board, Mami was selected as the favorite. All the students agreed that she is 
the loudest, clearest and easiest to understand. And one student added, " She is pretty." 
The Softball player was liked because he not only spoke in Japanese, but he also illus- 
trated his phrase by throwing a baseball. The students seemed to relate well to charac 
ters who were in the appropriate setting and were dressed to fit the part. e.g. the base- 
ball player was wearing a baseball uniform. 
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This group also identified the characters on the videodisc they did not particu- 
larly care for. The character who said Coca-Cola was not as appreciated because his 
statements were, according to one student, "too slow and sometimes too fast... and 
kinda weird." The karate person is not appreciated as well as others because he is not 
in a Karate uniform . One of the male students said, "He is not dressed in a uniform, a 
dojoe or anything." When the same student was asked if he liked the way the Karate 
person spoke he said that he did. 

Students identified how the teacher uses the video as a model. "If we do not 
understand something/ 7 one student said, "the teacher will play it again until we can." 
One student said she liked the vegetable section of the videodisc because it would 
show you the vegetables and then give you a couple of seconds to guess what it was. "I 
think this helped us to learn it a lot, memorize it and stuff," she added. 
They also agreed that the art work (Kangi) helped them to appreciate both the culture 
and the language. And they liked being able to paint their own Kangi. One girl said, 
"As I write it, it helps me remember it... "I think it's really beautiful and I like doing it." 
Students also like the games the teacher would use to help them learn. One game iden- 
tified involved coloring pictures of fruit an paper, cutting the pictures out, and placing 
them into an envelope. The instructor would then say the name of one of the fruits in 
the studenf s envelopes to see who could retrieve it the fastest The students liked the 
competitive nature of this game. 

When asked why they liked the videodisc, one student responded, "The video 
makes it so you can see the Kangi, and it is not from the teacher. "I like how you can 
actually see and hear people from the country say it," he said. 

Focus Gro up #2 morning class, March 4 

For the second day's 9:00 am class, we took six randomly selected students out- 
side for another focus group. This section reiterated the focus groups comments from 
the afternoon before. Students shared the same appreciation for characters. 

Mami was again chosen as the favorite character because, "she talks loud." The 
basket ball player was not liked by one student because, "He talks to fast and he does 
not talk clearly." The student added, "You cannot understand half the things he says." 
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When asked, if they would prefer instruction solely from the instructor, the students 
unanimously responded that they liked using the videodisc because the teacher did not 
know the language very well and because they enjoyed seeing a different culture. And 
the students liked seeing the characters from the disk in their natural settings. 

When asked if there was one thing they would do to make the videodisc better, 
one student said, "Put more people in it." She also said that it was the students on the 
disc that made her want to learn, "because grown-ups aren't as easy to understand." 
When asked if they would like to go to Japan on their vacations, the students all quickly 
responded with an enthusiastic, "yeah!" The students all agreed that class time went 
by very fast, because they were having fun learning. 

Classroom Observations, afternoon class March 4 

On the second visit to the 2:18 PM class, students learned their colors in Japanese 
through the use of the videodisc. Just as was the case on the first day in the 9:00 AM 
class students were all involved and eager to participate and vocalize their under- 
standing of the Japanese Language. This day was a reflection of the pervious days 
enthusiasm on behalf of both the teacher and her students. As an indication of the 
interest level of students one student tried repeatedly to re-enter the class after being 
dismissed for causing a disruption. 

Interview with Patricia Arroyo 

The teacher, Ms. Arroyo, was very vocal in her support for the videodisc pro- 
gram, and she seemed to approach our session prepared to give objective feedback on 
the videodisc. According to Ms. Arroyo, her students are learning faster than they nor- 
mally would as a direct result of the video. She said she's had guest speakers come and 
they are "amazed at how fast the students are picking it up." 

She said that she cannot use the videodisc for more than 15 - 20 minutes in each 
class period, because the students begin to loose interest. She said that she needs to 
keep the class lessons diversified; otherwise, she looses the attention of the students. 
She said that although the video is effective in conveying information, it does not con 
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tinue to hold the students attention beyond the 15 to 20 minute mark. She added that 
this is true of any school lessons. 

According to Ms. Arroyo, the ten teachers participating in this program in Clark 
County so far have been volunteers. They are people who are "gung ho" about the pro- 
gram. "Th; y all want to make it work," she said. I asked her if she thought the 
videodisc program would add to a class taught by a teacher who's not as motivated. 
She said that she did not know. 

Ms. Arroyo said that she believes a teacher who uses the video disk must have a 
language teaching background. "I think you need to be a language teacher." Teachers 
who use the disk need an understanding in language before they can teach using the 
videodisc." 

When asked if she would change anything in the videodisc program she said it 
would be the volume. "There are times when it is not loud enough. "The book is also 
darn near perfect too," she added. Although Ms. Arroyo had no problems using the 
videodisc remote control unit, it did take her some time to learn now to use it. She also 
indicated that she needed to replace the batteries often (Once per quarter) and that she 
would appreciate having a battery supply provided, because she is buying all of her 
own. 

Ms. Arroyo said that she enjoys learning along with the students and that the 
videodisc is helping her too. Asked if videodisc met her expectations. She said, "It sur- 
passed them. I thought it would be very intimidating." She said, "The disk makes it 
easy." Ms. Arroyo said other teachers were first baffled by the sounds of the language 
too. "...Now they are not, and it is the direct result of the videodisc." She would give 
the program an A. And if the scale was from 1 to 10 she said she would give it a 9, just 
because of some technical difficulties she has encountered. 

Ms. Arroyo said that while kids can be mean and will make fun of another cul- 
ture's dress codes and life styles, the videodisc helps address these problems. 
However, because children do not understand why different races look different, a 
brief lesson in anthropology would add understanding, because the technology of 
videodisc capability and its apparent impact on students. 
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Summary 

We believe the video disk is beneficial in the teaching of language. From our 
first observation to our last, we saw no student who disliked the program; actually, it 
was always just the opposite. The students were anxiously engaged in the learning 
process especially when the videodisc was being used. They thought that the videodisc 
was a great diversion from traditional teaching methods. 
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Students liked learning language from other children (characters from the 
videodisc) because they were able to identify with them. Having other youth model 
the language was useful. It helped to eliminate intimidation to see characters such as 
Mami in her country saying Japanese phrases. Where they were initially intimidated 
by the language, they all developed an appreciation which extended further titan just 
the language , to include the people and their culture. 

There are some characters in the program that could have been presented better 
such as the basketball and Softball players who were hard to understand. As a whole, 
showing the characters outside and involved with interests the children here in 
America have was a great influence. 

Technical problems such as calling up ice cream and seeing a picture of juice is a 
problem reported to have happened on occasion. Despite the programs few short com- 
ings, it is appreciated by students. They are learning Japanese better and enjoying it 
more than they expected they would. Most students indicated that it was their favorite 
class and that they would like to take another language class. 
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Introduction 

This is the summative evaluation report of the Japanese video disc inservice. To 
evaluate the inservice program, three approaches were adopted: (1) on site observa- 
tion; (2) questionnaire survey; and (3) telephone survey. The reason why two different 
surveys was conducted was that we thought that the questionnaires collected immedi- 
ate after the training might be different from the telephone survey which was conduct- 
ed almost 2 months later. 

This report consists of five major parts: 

(1 The first part is the notes of our observation of the three inservice 

training sessions. 
(2 The report of questionnaire survey. 
(3 The report of the telephone survey. 
(4 The strengths and possible improvements will be dis 

cussed. 
(5 The conclusion. 

The observation of the Inservice Sessions 

Three sessions of inservice training of one and one-half hours each were held 
over the ED-NET systems at various educational facilities around the state. The tech- 
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nology involved in the ED-NET broadcast was very impressive. It included: live video 
and audio from each of the sites, overhead camera shots for displaying the teachers 
manual, closed captions of the words or phrases being taught and video segment 
broadcasting. Based upon our team's viewing from the Logan site, there were only a 
few times when the originators experienced difficulties with the system. One instance 
was when a voice came across the air from an unidentified site, testing a microphone. 
The second was when the sound was missing from the video disc segment. Neither sit- 
uation was a serious distraction from the content of the session. 

The purpose of the sessions was to orient teachers to the use of Konichi Wa, to 
the technology, and answer the questions and concerns that the instructors might have 
while using the disc. The sessions were also designed to teach the classroom Japanese 
phrases and introduce the Japanese culture. 

The approach was to use a two-stage implementation process: first, to train a 
designated technical representative at each site and second, to inform all teachers who 
would use the videodisc program next year. The first session was designed to intro- 
duce the disc, its development and the content structure to the techinical representa- 
tives, as well as to train the specialists on how to use the disc and bar code reader. The 
idea behind familiarizing the specialists on how to use the video disc was to have them 
supervise the practice with the video disc player and bar code reader in the later ses- 
sions and assist the teachers on-site. It was planned that each site have two teachers 
function as specialists. However, almost half of the site specialists were not at the first 
training session. It appeared that nothing was done to compensate for their absence. 
The specialist at the Logan site didn't show up until the final session. 

The second session was designed to train the teachers who were going to teach 
Japanese with the video disc. At some of the sites, specialists were assisting those who 
needed help. The material taught at the second session was very similar to covered in 
the first session. Art Burnah, the course developer, and John McKee, a specialist in 
Japanese, alternated in teaching. Art gave an oveiview and introductions, while Dave 
taught the sample lessons. The teacher were also showed how to use the video disc 
player and bar code reader. 



Page 68 



Konichi Wa Evaluation Report 



Appendix F 



The third session was primarily focused on teaching the teachers to pronounce 
the classroom lapanese phrases correctly. The session also showed the Japanese cultur- 
al materials and how to use those materials to learn the language. Art and Dave 
answered the questions raised by the teachers. 

A total of 46 teachers had indicated that they would attend the inservice training 
However, the actual number of participants was less than expected. The third session 
was the only session when each site was represented with participants. Estimates of 
the number of actual attendees are as follows: April 18, (Technical Representatives only 
meeting) 18 people attended; April 15, 30-35 persons attended; and May 13, 35^0 per- 
sons attended. (Estimates based upon participant observers checking with State Office 
sources.) 



The Questionnaire Survey Results 



Twenty-five (25) participants returned the questionnaire survey sheets. Their 
responses of the inservice training program are generally positive. However, some of 
the participants also pointed out the weaknesses of the program which coincide with 
those possible improvements we suggest 



The following table shows the participants' rating of the program: 





RATING SCALE 




Strongly Agree 5 


4 3 2 


1 Strongly Disagree 



#of5's #of4's #of3's #of2's 



QUESTIONS NO RESPONSE 



Were your objectives met? 


8 


9 


5 


1 


2 


Do you feel that this inservice program 












prepared you adequately to use the disc in 












your classroom? 


9 


10 


6 






How was inservice organization? 


13 


6 


3 


1 


2 


How was introduction to teacher materials? 


15 


6 


2 


2 




How was teacher materials? 


16 


6 


2 




1 
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How was student materials? 
How was cultural materials? 
How was video segments? 
How was on-site practice sessions? 



#of5's #of4's #of3's #of2's 

13 9 2 1 

16 5 2 1 1 

18 6 l 

10 1 9 1 4 



From the table, we can see that most of the participants expressed positive atti- 
tudes toward the program. They commented that there was not enough hands-on 
practice. The ratings of the program appear uniformly high, which should be seen as 
no small accomplishment in delivering inservice instruction in a novel way to veteran 
teachers. THe two areas of some concern— and both show fairly high ratings— are their 
feelings of being adequately prepared for using the disc in the classroom and having 
enough on-site practice with the hardware. 

Telephone Survey Results 

Follow-up telephone calls were conducted between May 15th and June 15th, 
reaching 9 participants (see appendix G for the instruments used). 

Generally, the participants felt that the entire sessions were very helpful to them 
due to the nature of the setup and instruction provided. They felt that the program 
division was beneficial to their learning process because of how they were introduced 
to the various sessions. The practice of using the bar code reader was especially helpful 
to those who had never used one before. All of the participants thought highly of the 
practice of Japanese phrases. 

Most of the participants agreed that the repetitious nature of the inservice was 
instrumental to how they adapted to the lesson. They agreed that hearing the language 
from a non native speaker (John) gave them hope that the language can be learned. 
One of the participants felt that the inservice greatly helped in allowing her to see and 
communicate with other teachers. She felt that she could now have a contact base for 
future references. 
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Most of the inservice participants felt that the final session was really good and 
that the information presented was helpful. They felt the interactive nature of the ses- 
sion allowed them to freely raise any questions that m/.ght arise throughout the process 
whether in school or at the inservice session. The participants felt that the instruction 
was good due to its hands-on experience. They regarded such training as the focal 
point of their learning because it allowed them to practice things away from their stu- 
dents. 

Participants surveyed did not indicate any weak points of the inservice. They 
felt that it covered all bases and that it was beneficial to them. They mentioned that the 
inservice was generally good and wished it was done earlier. 

The inservice, according to the teachers, was very good. It allowed them to 
envision different ways to teach and allowed them a chance to see how language and 
culture are intertwined. They felt that the cultural display gave them a sense of the lan- 
guage itself. They wanted to learn more of Japanese culture. The participants felt that 
the instruction involved a lot of listening skills. That special education students do well 
as the other students in the class. 

One of the teacher mentioned liking the step-by-step instructional method to go 
through the sample lesson. Another participants thought that the cultural material 
W' re informational, and that it would be especially beneficial for the elemental y stu- 
dents who had limited knowledge of other cultures. 

Some of the participants were impressed with the videodisc and how well it had 
been received by the students had been exposed to it already. They felt that the inser- 
vice was "very adequate" in preparing them to use it in their class. They appreciated 
Art's willingness to help coordinate the grant request for matching funds. 

All the comments on the inservice were positive except one from a participant 
who complained about lack of practice of the disc because this participant's school did 
not have the video disc. 

Strengths of the Inservice Program 
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It was apparent that Art Burnah was very receptive to previous evaluations of 
his program. Two examples of his willingness to make suggested improvements are: 
1). Joan's suggestion that they explain the difference between two different forms of 
Japanese writing, which they did; and 2). It had been observed in Nevada that the 
teachers did not use Japanese expression to praise the students for their good work 
while teaching the lessons. Dr. Eastmond had suggested that they teach the teachers 
how to say the Japanese praises listed in the introduction to the manual so that the 
teachers would use them in the classroom. The Japanese praise phrases were practiced 
in the third session of the inservice training and were welcomed by all participants sur- 
veyed. 

The team teaching of Art Burnah and John McKee was perceived to be very 
effective. It was a good blend of overview and model teaching demonstrations. 

Possible Improvements 

Having a specialist at each site seemed to have a great impact on the success of 
the sessions, but almost half of the site specialists were absent at the first training ses- 
sion and it appeared that nothing was done to compensate for their absence. 

While in some cases the "technical specialist" may have been familiar with the 
equipment and/ or videodisc beforehand, some verification of readiness might have 
strengthened the technical representative's role. 

It might be helpful to have one person in charge at each site to ensure the teach- 
ing over the ED-NET was well received. One of the participants at the Logan site spent 
most of the time talking to one of the other teachers. This was quite distracting to oth- 
ers in attendance. Such communication between participants may be perceived as an 
advantage or as a disadvantage of teaching over a broadcast system. An organizer, or a 
supervisor, or even a specialist at the site may solve this problem. 

Based upon the interview feedback , the inservice training sessions may not 
have provided enough hands-on practice, otherwise it would be more successful. The 
real question, of course, is how well teachers will use the materials in class during the 
next school year. 
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Conclusion 

Overall, the inservice training program was rated very successful by partici- 
pants. It provided valuable information about the content of the Japanese videodisc, its 
lesson structure and cultural materials. It also provided some opportunities for the par- 
ticipants to learn how to use the video disc and the bar code reader. The practice of 
using Japanese Praise phrases helped some of the teachers break the barrier to open 
their mouths and speak a very unfamiliar language and gave them confidence in teach- 
ing the language. 

The program could be made more effective and efficient if the organization of 
the training at some of the sites was monitored more closely and if more hands-on prac- 
tice was provided for the participants. 
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Survey Schedule for Teachers 
Phone Survey for In-service Participants 
On Site Visit Form for Teachers 
On Site Visit Form for Focus Groups 
On Site Visit Form for Administrators 
Class Observation Form 
In-service Questionnaire 
In-service Information Sheet 
Inservice Phone Survey 
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Survey Schedule for Teachers 

concerning 
the Teaching of Konichi Wa 

I. Background Information 

Fill out the following information: 

Teachers name/ Home Room number: 

School's Name:_ 

Administrators (principle's) name: 

School District Questions: 

1. Have you had the opportunity to use Konichi Wa Japanese language video disc? 

2. If you have not, do you plan to use Konichi Wa in the future? 

3. If so, when? 

(If you have answered "no" to number one, please stop the survey here. Thank you.) 
II. Questions 

Instructions: Please answer the following questions as accurately as you can. If a selec- 
tion does not fit your analysis perfectly, please select the closest choice. At the end of 
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the survey, a section has been provided for your comments. We appreciate your help 
in completing this survey. 

Please note: When the term "administrator is used, the survey refers to the school prin- 
ciple or director, not to the school administrator. 

Please respond by marking the appropriate selection: 

1. On a scale from one to five, five being the most confident, how would you rate your 
confidence level as a teacher using Konichi Wa? 

5 - extremely confident all the time 

4 - confident most of the time 

3 - confident at times 

2 - not too confident 

1 - not at all confident 

2. How would you rate the confidence level of your students in using Konichi Wa? 

5 - extremely confident all the time 

4 - confident most of the time 

3 - confident at times 

2 - not too confident 

1 - not at all confident 

3. How much do you use the manual for Konichi Wa? 

all the time 

most of the time 

sometimes 

rarely 

„ not at all 
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4. Do you use each iesson from beginning to end? 

Yes No 

5. In your opinion, your students are about using the language videodisk. 

really excited 

somewhat excited 

a little bit excited 

not excited 

not at all excited 

6. What aspects of the program seem to be most intriguing to the students (if any)? 

language learning 

cultural learning 

the television 

learning on their own 

other (please specify) 

7. What aspects of the program seem to be distracting to the students (if any)? 

the equipment 

the lack of teacher instruction 

other (please specify) 

nothing is distracting 

8. If the equipment distracts, it distracts because of... 

the novelty of the equipment 

my inability to utilize the equipment 

faulty equipment 

other (please specify) 
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9. What skills are the students learning from the program? 

verbal 

cultural knowledge 

grammer 

other (please specify) 

10. Overall, in terms of student living, how effective is the program? 

very effective 

somewhat effective 

not too effective 

not at all effective 

I don't have any idea 

11. On a scale from 1 to 4, 4 being the most adequate, how would you rate your 
knowledge of the language you are teaching? 

very adequate 

adequate 

somewhat adequate 

not at all adequate 

12. Do you have technical problems when using the equipment? 

virtually all the time 

some of the time 

rarely 

not at all 

not sure 
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13. If you have any technical problems, what kinds of problems are they? 



14. Does Konichi Wa fulfill your needs as a teacher in teaching this course? 

Yes No Too early to tell 

15. How easily can you adapt/ change the course to fit your teaching needs? 

very easily 

easily 

^pretty easily 

not easily at all 

16. What was your preconception of Konichi Wa? 



17. Now that you have worked with the language videodisk program, how does your 
preconception match the reality? 



18. How often is Konichi Wa used in your classroom by the same class? 

5 or more times a week 

3 to 4 or more times a week 

1 to 2 or more times a week 

1 time every couple of weeks 

unsure 
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19. Do you think your ability/ inability to use the videodisk equipment effects your 
confidence in teaching the language? 

Yes No I'm not sure 

20. Do you think you are utilizing the program to its maximum potential? 



21. If no, why not? 



i 
i 



i 
i 

i 
i 



22. If so, what help you ultilize the program to its full potential? 



23. Overall, in terms of caching potential, how efficient is the program? 

very efficient 

somewhat efficient 

not too efficient 

not at all efficient 

I don't have any idea 



24. How much time during each class period do you use Konichi Wa? 



_45-50 minutes 
_30-45 minutes 
J. 5-30 minutes 
5-15 minutes 



25. Are the lesson appropriate for 45-50 min. sessions (especially for middle school)? 
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26. In teaching language, I use the videodisk as ? 

my main teaching instrument 

a supplemental instrument 

27. How much time do you communicate/ interact with students while using Konichi 
Wa? 

45-60 minutes 

30-45 minutes 

15-30 minutes 

5-15 minutes 



28. How much do you communicate/ interact? (Mark all that apply) 

verbal instruction 

prompt students to participate 

verbal correction 

interruption of the program 

body language 

other (please specify) 

29. How do students participate in the program? (Mark all that apply) 

verbal repetition 

view the video 

write words 

body language 

follow instructions from video 

follow instructions from teacher 

other (please specify) 
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30. What are your teaching objectives in using Konichi Wa? 



31. Do you get any input from outside sources which influence the way you use 
Konichi Wa? 

Yes No 

32. From where/whom? 

33. Whose idea was it to use Konichi Wa? 



.administrators 
_school district's 
.parents' 
.students' 

_other (please specify) 



34. Do you have a choice as to whether or not you use Konichi Wa? 

Yes No 

35. Do you have a choice to what extent you use Konichi Wa? 

Yes No 

36. Do you use other materials to supplement or enrich Konichi Wa? 

Yes No 
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37. If so, what? 



38. How do you determine whether your students are learning or not? 



_pre-test 

.post-test 

_pre- and post-test 

.other (please specify) 



39. Have you used videodisks in your teaching before using Konichi Wa? 

Yes No 

40. If so, what subject(s)? 



ERIC 



41. How much computer experience did you have before first using Konichi Wa? 

several years 

several months 

a few weeks 

a few hours 

none whatsoever 

42. Do you feel more previous computer training would have affected your confidence 
level teaching? 



43. In which way? 

positively negatively 
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44. Do you feel more previous computer training would have effected your confidence 
level in utilizing the equipment? 

Yes No 

45. In which way? 

positively negatively 

46. The training I did have my ability to learn how to teach Konichi Wa? 

did not help helped hindered neither helped nor hindered 



47. Did you have any exposure to the language(s) being taught before utilizing Konichi 
Wa? 

I was fluent in the language 

I could accomplish basic conversation 

I knew and could use a few words 

I had regularly heard the language, but had no under 

standing 

48, If you were exposed to the language before using the language video disc program, 
on a scale of 1 to 5, 5 having the most effect, how did your previous exposure affect 
your confidence in teaching Konichi Wa? 

5 - highly affected 
4 - very affected 
3 - affected 
2 - not too affected 
1 - not at all affected 
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49. How much time outside of class was needed to prepare yourself and the materials? 

too much time 

considerable time 

acceptable amount of time 

not much time 

very little time 

50. Have you used the teacher explanations found in the support materials? 

Yes No 

51. If not, why? 



52. If so, did you find them clear and helpful? 



53. Have you received any feedback from parents regarding Konichi Wa? 

Yes No 



54. If so, what feedback have you received? 
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55. Have you used any kind of assessment tests to determine language/cultural learn 
ing? 

Yes No 

56. If so, did you use your own tests? 

57. If not, what kind of assessments have you done? 

i 

58. Overall, the program Konichi Wa is an excellent teaching tool. 

strongly agree 

agree 

unsure 

disagree 

strongJv disagree 

59. If your answer is other than unsure, please state why: 



III. Comments: 
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Phone Survey: In-service Participants 

1. Introduce yourself 

2. Could you describe the in service training you received? 

3. What were you hoping to learn from the in service? 

Use of apparatus? 

Language teaching techniques? 

i. Was the attendance by choice or was it required by your school? 

5. What was your opinion of the actual teaching materials? 

Age appropriate? 

Visual appeal? 
Easy to use? 

6. How did you make use of the in service experience in your district or classroom? 

7. Thinking back, how would you improve the in service experience? Are there other 
things you feel you should have learned in the in service? 
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8. Do you have any final comments? 



For those who did not use the Videodisk: 

What was the main reason for not using the video disk? 
a-You need more training with the equipment? 
b-You needed more training with the language? 
c-Were there logistic difficulties? 
d-Scheduling problems? 
e-Too busy? 
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On Site Visit - Teachers 

1. Teacher's Name: 

2, How did you first hear about Konichi Wa? 

3, Please describe your class (size, grade, learning abilities, special challenges). 

4. What was the reaction of the students to this type of learning? 

High achievers? 

Low Achievers? 



Those with handicaps(physical or mental)?. 
Does the pedagogy work for the students?. 



Can they perform the lesson tasks? 

Does the program maintain their interest? Why or why not?. 
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5. Write your own question time: Use the surveys (if you have them) at this point 
Ask a few questions, probing to let them know you are following up on the survey . 
Especially pay attention to the written comments they put down. For example, one 
teacher might have said that he did not feel he was using the program to its maximum 
potential because of his large class. Some students have high interest. Others very little 
interest. It's difficult for him to keep a learning atmosphere. A possible probe might 
be: " why are some of your students more interested in the lesson than others?" or 
"What do you feel could help you maintain a learning atmosphere with regards to the 
Konichi Wa program?" 

First Question: 



Second Question: 



Third Question: 



Fourth Questions 
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6. Thinking back what sort of in service did you have? 
How helpful was it? 



Was there any additional support? 



7. Would you use this program when it is available in French, Spanish, Chinese, or 
Korean? How would you use it differently than Konichi Wa? 



8. Do you have any suggestions for the people developing this videodisk and training? 



9. What kind of hardware was available to you? How easy was it to get the equipment 
you used? Did you face any competition for its use? 



10. Do the manuals and bar codes work well? 
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11. Have you received any feedback from parents? 

12. What have been your personal benefits, if any from this program? Are you 
learning the language? 

13. What is your overall reaction to Konichi Wa? 

Would you use it again? 
Is it easy to use? 
Is it time efficient? 
Do you feel in control? 

14. Any final comments? 
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On site visit - Focus Groups 

The intent here is to foster group discussion. If necessary, call on students to get a 
response. 

Students will be selected by looking at the roll book and starting with the 4th kid, select 
every — th one. (Enough kids to get 6 kids) For example, if the teacher has 30 kids, 
select every 5th one after the 4th. 

One person will be the discussion leader. The other should observe and record. 
(Remember you will have the video camera for back-up.) 

Rapport Building - Mention why they are there and that you will have a video camera 
on them. Then play the name game: 

First student will state his/her name and what he/ she likes to do. The next stu- 
dent will repeat what the first said and tell about him/ herself. And so on around the 
group. The discussion leader will be the last one. (Note, one member of the team will 
be the discussion leader and the other one will be a recorder.) 

1. How long have you studied Japanese? 

2. How doe you like Konichi Wa? Who is you favorite character? Whom do you like 
the least? 

3. What have you learned? 

-ask individuals for words and phrases they know (go around the circle) 

-call on each person and note willingness to respond (the recorder should keep 

track of the students who are as well as those that can respond. This should go 

into the report) 

-ask about any cultural things they may have learned. 

4. How do you learn Japanese best? How did you learn the material for today's lesson? 
(mention what it was about and be specific with the day's lesson) 
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5. Can you think of ways to make this type of learning better? 

On-site visit - Administrators 

Preliminaries: Introduce yourself and the project you are representing. Name the 
teachers you will be visiting. Talk about the procedures being used in the classroom. 
The ask the principle the following questions: 

1. What has you involvement been with the Japanese video disc project? 

Have you observed using it? 

Have you seen the materials? 

2. In general, are you in favor of foreign language work at the grade school level? 

How do you feel language training should be handled? 

3. What has been your impression of the Konichi Wa program? 

Does the videodisc seem effective? 

Does it seem time-consuming or bothersome? 

Would you recommend other teachers use it? 

4. Have you had any parental feedback? 

Has it been positive or negative? 

5. From what you know of the program (all aspects), how do you think it could be 
improved? 
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Class Observation Form: Team Member 1 

Preliminaries: 

How Many Students? How Many Girls How Many Boys 

Is the teacher fluent in the language being taught? 

During the Discussion: 

1. How well are different types of learners performing? Note attitudes, accomplish- 
ments, language skills 

2. What learning strategies are being used? 

General Specific (for specific units) 

3. What would you do to improve the effectiveness of those learning strategies? 



Konichi Wa Evaluation Report Page 97 

93 



Utah State Evaluation Team 



Japanese Videodisc Inservice 
Questionnaire 

As part of the evaluation of the Japanese Videodisc Project, we are looking at ways of 
making inservice training more effective. Please respond to the following items in writ- 
ing or by circling the most appropriate answers (S=high, 1 =low). Your responses will 
be treated confidentially, with results reported as group data. 



1. What were your objectives / expectations in attending this 
inservice? 



met or exceed - 



adequate 



3. Do you feel that this inservice program 
prepared you adequately to use the disc 
in your classroom? 



high quality 



did not 







ed 






meet 


2. 


Were your objectives met? 


1 


2 


3 4 


5 



inadequate 



1 


2 


3 


4 


5 



low quality 



4. Please rate the following components of 
the overall program: 

INSERVICE ORGANIZATION 

INTRODUCTION TO TEACHER 
MATERIALS 

TEACHER MATERIALS 



1 


2 


3 


4 


5 




1 


2 


3 


4 


5 




1 


2 


3 


4 


5 



ERIC 
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STUDENT MATERIALS 
CULTURAL MATERIALS 
VIDEO SEGMENTS 
ON-SITE PRACTICE SESSIONS 



1 


2 


3 


4 


• I 


1 


2 


3 


4 


5 




1 


2 


3 


4 


5 




1 


2 


3 


4 5 



5. Describe your current impressions about the following: 

INSERVICE PROGRAM 
VIDEO DISC MATERIALS 
USE OF THE EQUIPMENT 

6. Please make any additional comments on the back of this form. Thank you for your 
help! 
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JAPANESE VIDEO DISC INSERVICE 

INFORMATION SHEET 

The purpose of this evaluation is to solicit feedback concerning the inservice program 
for future improvements. A random telephone survey will also be conducted with 
some of you following the final session. Please fill out the information sheet and indi- 
cate when it would be most convenient to contact you. Please leave these completed 
forms with a person at your inservice sight. Thank you. Your cooperation is greatly 
appreciated. 

Name Female/ Male 

School Grade level/ s 

Subjects taught ; 

Foreigh Language Experience 



Video Disc Experience (circle one) NONE SOME A LOT 
When would it be convenient for us to contact you? 

Week Day Times Phone 
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TELEPHONE SURVEY 

NAME: 

SCHOOL: 

DATE: 

Is this a convenient time to talk? 

If not, when would be a better time I could call 

back? 



Establish Rapport-Expiain the reasons for the study, the connection with USU, and that 
responses will be kept confidential for any specific individual. Then ask: 

1. Could you describe how you learnea ^bout the Japanese 
Videodisc Project? 



2 Now that you have experienced the inservice training, 

a. How adequate did you find the training in meeting your 
expectations? 



Probe: (Strong Points). 
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(Weak Points). 



(Ed Net). 



(On-site practice session) 
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